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(54) Image Information encoding/decoding system 

(57) A 6ut>picture packet is encoded using a packet 
header (PH) containing a time stamps sub-picture data 
(32) containing compressed pixel data (PXD), one or 
mare display control sequences (33), and a sub-picture 
unit header (31). The lime stamp indicates the playback 
start time of the sub-picture packet The display control 
sequence (DCSQT) defines the order oi display of the 
sub-picture data (PXD). The sub-picture unit header 
(SPUH) indicates the size of the sub-picture packet and 
the location of the display control sequence (DCSQT). 
The encoded sub-picture packet is decoded, and dis- 
played based on the cfi^lay control sequence 
(DCSQT). 
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Description 

The present invention relates to an improvement of 
a system for encoring and decoding image information 
such as a sib-picture which is reproduced or played 5 
back simultaneously with a main picture. 

The present invention also relates to a recording 
medium such as an optical disk and a data recording 
scheme for recording sub-picture data supplied 
together with picture data on the recording mecfum and 
simultaneously reproduced with the picture data from 
the recording mecfum. 

Sub-pictures superimposed on main pictures, such 
as titles in a movie and fenage contents to be displayed 
as a set value of the television volume level are mainly 
realized by two schemes, i.e., a character code scheme 
and a bit map data scheme. 

in the character code scheme, characters such as 
pre-registered and prepared characters and patterns 
are held in a character recoiding area of a character 
generator. A code assigned to each character is input to 
the character generator to display a desired character. 

According to this scheme, dedicated hardware 
such as a character generator is reqiired, but the vol- 
ume of data to be sent to a display system can be 
srnayer than that of a scheme for directly sending char- 
acter bit map datatolhe display system so as to display 
a sub-picture, because a code is input to display a char- 
acter. However, the applications of display of sub-pic- 
tures accortfrig to this scheme are limited since the pre- 
registered and prepared characters can only be dis- 
played 

To the contrary, in the bit map data scheme, dedi- 
cated hardware for generating a sub-picture from a 
code need not be used, because the bit map data of the 
sub-picture is cfirectiy sent to a display system. The sub- 
pictures can be used in a variety of display applications 
since the form of the sub-picture to be displayed is not 
fixed. 

According to this method, sub-picture color data, 
sii>-picture outline color data required for siper-impos- 
ing a sub-picture on a main picture, and super-imposing 
mixing ratio data of the main picture and the sub-picture 
must be provided for each pixel. Therefore; the volume 
of data to be sent to a display system is inevitably very 
large. In addition, in the bit map data scheme, data 
associated with aD pixels on a display screen (to be 
referred to a frame hereinafter) must be sent to the dis- 
play system regardless of the size of a sub-picture (see 
FIG. 50). For this reason, a large amount of wasteful 
data are present in respect of cfcptay space. 

In either the character code scheme or the bit map 
data scheme, even if the shape of a sub-picture being 
currently displayed does not change, sub-picture data 
must be basically kept sixplied for each display frame 
period, and a large amount of wasteful data are present 
with respect to the display time. 

The present invention has been made to solve the 
conventional problems described above, and has its 



object to provide a data recording method and a record- 
ing medium, in which sub-picture data wasteful in 
respect of both display space and display time can be 
greatly reduced, the degree of freedom in sub-picture 
expressions is large, and a variety of applications of 
sub-pictures can be assured. 

bi order to achieve the above object, according to 
the first invention, there is provided a data reconcfing 
method of forming, in a predetermined unit a packet of 
sub-picture data simultaneously reproducible with main 
picture data, and record ng the sub-picture data packet 
on a recording medium, wherein the sub-picture data 
packet includes at least packet header information (PH) 
including a playback start time of the sub-picture data 
packet which is expressed using a reference time, sub- 
picture content data (PXD) representing the display 
contents of a sub-picture, at least one display control 
sequence information (DCSQT) representing a control 
sequence tor displaying the sub-picture using the sub- 
picture content data, and sub-picture header informa- 
tion (SPUH) including a size of the sub-picture data 
packet and a recording position of the display control 
sequence information, and the sub-picture data packet 
is recorded on the recording medium. 

According to the second invention, there is provided 
a data recorcfng method of forming, fo predetermined 
units, packets of sub-picture data of a plurality of chan- 
nels simultaneously reproduce with main picture 
data, respectively, and recording a plurality of data 
packets of a selected one of the plurality of channels as 
one sub-picture data block on a recording medium, 
wherein the sub-picture data packet includes at least 
packet header information (PH) inducing a playback 
start time of the sub-picture data packet which is 
expressed using a reference time, cub-picture content 
data (PXD) representing the display contents of a stfo- 
picture, at least one tf splay control sequence informa- 
tion (DCSQT) representing a control sequence for dte- 
ptaying the sub-picture using the sub-picture content 
data, and sub-picture header information (SPUH) 
including a size of the sub-picture data packet and a 
recording position of the display control sequence infor- 
mation, the sii>-picture data packet is recorded on the 
recording medium, and the playback start lime (PTS) of 
the sub-picture data packet in the packet header infor- 
mation is recorded only for a start sub-picture data 
packet in the sub-pictire data block. 

According to the third mention, there is provided a 
data recording method as defined in the first or second 
invention, wherein the display control sequence infor- 
mation (cf. DCSQ in FIG. 33) has at least a sub-picture 
display start time, a sub-picture display end time, the 
recording position of the sub-picture content data to be 
displayed, and a group of display control information for 
the si4>-picture content data recorded at the recording 
position. 

According to the fourth invention, there is provided 
a data recording method as defined in the third inven- 
tion, wherein the sub-picture display start lime and the 
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sub-picture display end time are defined as times rela- 
tive to the playback start time of the sub-picture data 
packet. 

According to the fifth inventor, there is provided a 
data recording method as defined In the third invention, 
wherein display start control information (cf. STA_DSP 
in FIG. 34) for controlling a start of display of the sub- 
picture content data is recorded as the display control 
information on the basis of the display start time 
included in the display oontrol sequence Information. 

According to the sixth invention, there is provided a 
data recording method as defined in the third invention, 
wherein display end control information (cf. STP.DSP 
in FIG. 34) for controlling an end of display of the sub- 
picture content data is recorded as the display control 
information on the basis of the display end time included 
in the display oontrol sequence information. 

According to the seventh invention, there is pro- 
vided a data recording method as defined in the third 
invention, wherein color setting information (cf. 
SET_COLOR in FIG. 34) for setting a color of the sub- 
picture content data for each pixel type is recorded as 
the display control information. 

According to the eighth invention, there is provided 
a data recording method as defined in the third inven- 
tion, wherein mixing ratio setting information (cf. 
SETCONTR in FIG. 34) lor setting a mixing ratio of the 
main picture and the sub-picture content data for each 
pixel type is recorded as the display control information. 

According to the ninth invention, there is provided a 
data recording method as defined in the third invention, 
wherein display area setting information (cf. 
SET.DAREA in FIG. 34) for setting a display area of the 
sub-picture content data on the maki picture is recorded 
as the display control information. 

According to the tenth invention, there is provided a 
data recording method as defined in the third invention, 
wherein use range setting information (cf. SETJ)SPXA 
in FIG. 34) for setting a range to be used in a display in 
the sub-picture content data is recorded as the display 
oontrol Information. 

According to the eleventh invention, there is pro- 
vided a data recording method as defined in the third 
invention, wherein color/mixing ratio change setting 
information (cf. CHGLCOLCON in FIG. 34) for setting, 
in units of pixels, changes In a color of the sub-picture 
content data for each pixel type and in a mixing ratio of 
the main picture and the sub-picture content data for 
each pixel type is recorded as the display control infor- 
mation. 

According to the twelfth invention, there is provided 
a data recording method as defined in the third inven- 
tion, whereto display control end setting information (cf. 
CMD_END in FIG. 34) for setting an end of display con- 
trol for the sub-picture content data is recorded as the 
display control information. 

Accartfing to the thirteenth invention, there is pro- 
vided a recording mecSum on which a packet ol sub-pic- 
ture data simultaneously reproducible with main picture 



data is formed in a predetermined unit and recorded, 
wherein the sub-picture data packet includes at least 
packet header information (PH) including a playback 
start time of the sub-picture data packet which is 

s expressed using a reference time, sub-picture content 
data (PXD) representing the display contents of a sub- 
picture, at least one display control sequence informa- 
tion (DCSQT) representing a control sequence for dis- 
playing the sub-picture using the sub-picture content 

10 data, and sub-picture header information (SPUH) 
inducing a see of the sub-picture data packet and a 
recording position of the display oontrol sequence infor- 
mation. 

According to the fourteenth invention, there is pro- 
is vided a recording medum on which packets of sub-pic- 
ture data of a plurality of channels simultaneously 
reproducible with main pictire data are formed in prede- 
termined units, respectively, and a plurality of data pack- 
ets of a selected one of the plurality of channels are 
20 recorded as one sub-picture data block, wherein the 
sub-picture data packet includes at least packet header 
information (Phi) inducing a playback start lime of the 
sub-picture data packet which is expressed using a ref- 
erence time, sub-picture content data (PXD) represent- 
as ing the display contents of a sub-picture, at least one 
display control sequence information (DCSQT) repre- 
senting a control sequence for displaying the sub-pic- 
ture using the sub-picture content data, and sub-picture 
header Information (SPUH) including a size of the sub- 
so picture data packet and a recording position of the rjs- 
play oontrol sequence information, and the playback 
start time (PTS) of the sub-picture data packet in the 
packet header information is recorded only tor a start 
sub-picture data packet in the sub-picture data block. 
ss According to the present invention, since one or 
more pieces of the display control sequence information 
(DCSQT) is used, sub-picture data wastefii in respect 
of both display space and display time can be greatly 
reduced. At the same time, the degree of freedom in 
40 sub-picture expressions equivalent to the bit map data 
scheme can be achieved, and a variety of applications 
of the 6ub-ptcture6 can be assured. 

More specifically, according to the present inven- 
tion, the use range setting information (SET_DSPXA) 
46 for setting the range to be used for a display in the sub- 
picture content data is recorded so as not to display 
data falling outside the use range. Wasteful data in 
respect of display space, which are created in transmit- 
ting one-frame data to the display system, can be 
so greatly reduced. 

In addition, accord in g to the present invention, the 
color setting information (SET.COLOR) and the mixing 
ratio setting information (SET.CONTR) tor each pixel 
type of sub-picture content data, such as a pattern pixel, 
55 an outline, and a background, are recorded to obtain 
sub-picture content data as information representing 
the shape of a sub-picture image. A sub-picture shape 
expression almost equivalent to the conventional 
scheme for providing color information and mixing ratio 
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information for each pixel can be guaranteed in a 
smaller volume of data. 

Furthermore, aocorcfing to the present invention, 
the color/mbdng ratio change setting information 
(CHG.COLCON) for setting, in urate of pixels, change s 
in the color of the sub-picture content data for each pixel 
type and in the mixing ratio of the main picture and the 
sub-picture content data for each pixel type is recorded. 
A sub-picture can be dynamically displayed with an 
accuracy equal to that of the conventional bit map data io 
scheme, but in a smaller volume of data than that 

Incidentally, the color information of a sub-picture 
rarely changes in unhs of pixels, and the data volume of 
the colorAnixing ratio change setting information is not 
expected to increase excessively. 75 

In addition, according to the present invention, even 
If the color of a sub-picture image changes, the sub-pic- 
ture can be displayed across a plurafity of frame times 
using the same sub-picture content data as long as the 
shape does not change. As compared with the conven- so 
tionai scheme wherein the sub-picture data is kept sup- 
plied to the display system at the frame period 
regardless of changes in color and shape of the sub-pic- 
ture, wasteful sub-picture data in respect of display 
space can be greatly reduced. & 

This invention can be more fully understood from 
the following detailed description when taken in con- 
junction with the accompanying drawings, in which: 

FIG. 1 is a view showing the structure of data ao 
recorded on an optical disk as an information hold- 
ing mecfium to which the present invention can be 
applied; 

FIG. 2 is a view showing the logical structure of 
data to be recorded on the optical cfiskin FIG. 1; as 
FIG. 3 is a view showing the logical structure of a 
sub-picture pack to be encoded {run-length com- 
pressed and supplied with a display control 
sequence table), of the data structure shown in 
FIG. 2; 40 
FIG. 4 is a view showing the contents of a sub-pic- 
ture data, of the sub-picture pack in FIGL 3, to which 
an encoding method according to an embodiment 
of the present invention is applied; 
FIG. 5 is a view for explakiing concession rules 1 4S 
to 6 used in an encoding method according to an 
embodiment of the present invention in a case 
wherein Image data constituting the sub-picture 
data portion in FIG. 4 consists of a plurality of bits (2 
bits in this case); so 
FIG. 6 is a view for explaining compression rules 11 
to 15 used in an encoding method according to 
another embodiment of the present invention in a 
case wherein image data constituting the sub-pic- 
ture data portion in FIGL 4 consists of 1 bit; ss 
FIG. 7 is a view for explaining a detailed example of 
how the pixel data of each line is encoded (run- 
length compressed) in a case wherein pixel data 
constituting the sub-picture data portion in FIG. 4 



consists of, eg., first to ninth lines, 2-brt pixels (a 
maximum of four types) are arranged on each line, 
and character patterns "A" and m B m are expressed 
by the 2-bit pixels on the respective lines; 
FIG. B is a view for explaining two examples (non- 
interlaced display and interlaced display) of how the 
character pattern "A" of the pixel data (sub-picture 
data) encoded as shown in FIG. 7 is decoded; 
FIG. 9 Is a view for explaining compression rides 1 
to 6, in detail, which are used in an encocfing 
method according to an embodiment of the present 
invention in a case wherein image data constituting 
the sub-picture data in FIG. 4 consists of 2 bits; 
FIG. 10 is a block dia^am for explaining the flow of 
processing from the step of mass-producing a high- 
density optical disk taaving image information 
encoded on the baste of the present invention to the 
step of playing back the i n f o rm a t io n on the user 
side, and the flow of processing from the step of 
broadcast* ngtoHe-d retributing of image informa- 
tion encoded on the basis of the present Invention 
to the step of receiving^] aying back the information 
on the user/sub-pictire side; 
FIG. 11 is a block diagram lor explaining an enixxJ- 
iment (non-interlaced specification) of decoder 
hardware for executing image decoding (run-length 
expansion) according to the present invention; 
FIG. 12 is a block cfagram for explaining another 
embodiment (interlaced specification) of decoder 
hardware for executing image decoding (run-length 
expansion) according to the present invention; 
FIG. 13 is a flowchart for explaining software for the 
execution of image encoding (run-length compres- 
sion) according to an embodiment of the present 
invention, which is executed, for example, by an 
encoder (200) in FIG. 10; 

FIG. 14 is a flow chart for explaining the contents of 
encocfing step 1 (STB06) used in the software in 
FIG. 13; 

FIG. 1 5 is a flow chart for explaining software for the 
execution of image decoding (mn-length expan- 
sion) according to an embodiment of the present 
invention, which is executed by an MPU (112) in 
FIG. 11 or 12; 

FIG. 16 is a flow chart for explaining the contents of 
decoding step (ST1005) used in the software in 
FIG. 15; 

FIG. 17 is a block cfiagram for explaining another 
embodiment of the decoder hardware by which the 
Image decoding (run-length expansion) according 
to the present invention is executed; 
FIG. 18 is a flow chart for explaining the former 
processing of the image decoding (run-length 
expansion} acooiding to another embodiment of the 
present invention; 

FIG. 19 is a flow chart for explaining the latter 
processing (subsequent to node A in FIG. 18) of the 
image decoding (run-length expansion) accorcing 
to another embodiment of the present invention; 
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FIG. 20 is a flow chart for exemplifying the content 
of the coding header detection step ST1205 shown 
in FIG. 18; 

FIG. 21 is a flow chart for explaining the image 
decode processing according to the present invar- 5 
tion, wherein the decoded image is scrolled; 
FIG. 22 is a block diagram lor explaining a case 
wherein the compressed data, reproduced from a 
high-density optical disk having image information 
encoded according to the present invention, is io 
cfirectiy on-aired or output to a cable, and the on- 
ared or cable-distributed compressed data is 
decoded at the user side or at the subscriber side; 
FIG. 23 is a block diagram lor explaining a case 
wherein image in formati on encoded on the basis of is 
the present invention is exchanged between two 
arbitrary computer users via a communication net- 
work (e.g., an internet); 

FIG. 24 shows a brief configuration of a 
recordfplayback apparatus for recording on optical 20 
disk OD the image information encoded according 
to the present invention, and for playing back the 
recorded information to decode it according to the 
present invention; 

FIG. 25 shows an example of an IC device in which 25 
the encoder of the present invention is integrated 
with its peripheral circuits; 
FIG. 26 shows an exarrple of an IC device in which 
the decoder of "the present Invention is integrated 
with its peripheral circuits; so 
FIG. 27 shows an exarrpie of an IC device in which 
the encoder and decoder of the present invention 
are integrated with their peripheral circuits; 
FIG. 28 explains a position of time starry (PTS) in 
the data block of sub-pkftre; ss 
FIG. 29 explains a data structure of the data packet 
of sup-picture; 

FIG. 30 explains a relation between series sub-pic- 
ture units and the time stamp (PTS) as well as the 
display control sequence (DCSQ) which are *> 
described in the packet header of one of the series 
sub-picture units; 

FIG. 31 explains the size of sub-picture as well as 
the start address (relative address pointer of 
DCSQ) of display control sequence table, which 4s 
constitute a part of parameters contained In the 
sub-picture unit header (SPUH) of FIG. 3 or FIG. 4; 
FIG. 32 explains a configuration of a sub-picture 
display control sequence table (SPDCSQT); 
FIG. 33 explains contents of respective parameters bo 
(DCSQ) constituting the table (SPDCSQT) of FIG. 
32; 

FIG. 34 explains contents of display control com- 
mand (SPDCCMD) of sub-picture; 
FIG. 35 explains contents of pixel control data ss 
(PCD); 

FIG. 36 shows a bit-configuration of command 
FSTAl.DSP, which is one of the commands exempli- 



fied in FIG. 34, for forctoly setting the display dart 
timing of the pixel data of sub-picture; 
FIGL 37 shows a bit-configuration of command 
S1AJ3SP, which is one of the commands exempli- 
fied in FIGL 34 ( for setting the display start tinting ol 
the pixel data of sub-picture; 
FIG. 38 shows a bit-configuration of command 
STPJ3SP, which is one of the commands exempli- 
fied in FK3. 34. for setting the display stop timing ol 
the pixel data of sub-picture; 
FIG. 39 shows a bit-configuration of command 
SET_COLOR, which is one of the commands 
exemplified in FIG. 34, for setting the color code of 
the pixel data of sub-picture; 
FIG. 40 explains an exarrple of color data process- 
ing performed in the sub-picture data processor 
(e.g., decoder 101 in FKSL 11); 
FIGL 41 shows a bit-configuration of command 
SETjCONTR, which is one of the commands 
exemplified in FIG. 34, lor setting the contrast of the 
sub-picture with respect to the main picture; 
FIG. 42 shows a bit-configuration of command 
SET_DAREA, which is one of the commands exem- 
plified in FIG. 34, for setting the display area of the 
pixel ctela of sub-picture; 

FIG. 43 shows a bit-configuration of command 
SETJ3SPXA, which is one of the commands exem- 
plified in FIG. 34, for setting the display mart 
address of the pixel data of subpicture; 
FIG. 44 shows a bit-configuration of command 
CHG__COLCON, which Is one of the commands 
exemplified in FIG. 34, for setting the display area of 
the pixel data of sub-picture; 
FIG. 45 shows a bit-configuration of command 
CMD_END, which is one of the commands exempli- 
fied in FIG. 34, for terminating the display control ol 
sub-picture; 

FIG. 46 shows a brt-cortfiguration of line control 

inf or mat io n LCINF for pixel lines, which is one of the 

parameters exemplified in FIG. 35; 

FIG. 47 shows a bit-configuration of line control 

information LCINF for pixel lines, which is one of the 

parameters exemplified in FIG. 35; 

FIG. 48 shows an example of the displayed frame of 

sub-picture; 

FIG. 49 shows contents of respective parameters of 
pixel control data (PCD) of FIGL 35, when the cfis- 
play frame of subpicture includes a pixel pattern as 
shown in FIG. 48; 

FIG. 50 explains a problem occurred when the bit- 
mapped data of sub-picture is processed withoul 
use of the present invention; 
FIG. 51 firth er explains another problem occurred 
when the bit-mapped data of sub-picture is proc- 
essed withoii use of the present invention; 
FIGL 52 explains how the state of storing sub-pic- 
ture data blocks is changed with sub-picture chan- 
nels each having time stamp (PTS), when the sub- 
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picture data is decoded according to the present 
invention; 

FIG. 53 is a flowchart tor explaining an exarrple of 
encoding the sub-picture accord ng to the present 
invention, wherein the explanation is chiefly 5 
directed to the processing of a display control 
sequence (DCSQ); 

FIG. 54 is a flowchart for explaining an example of 
parallel-processing of the pack-disasserrtsiing of 
sub-picture data stream end decoding thereof, 
which data stream is encoded acconcfing to the 
process of FIG. 53; 

FIG- 55 is a flowchart for explaining an example of 
the pack disassemble processing of FIG. 54; 
FIG. 56 is a flowchart for explaining an exanple of 
the sub-picture decode processing of FIG. 54; 
FIG. 57 is a flowchart for explaining another exam- 
ple of parallel-processing of the pack-disassem- 
bling of sub-picture data stream and decoding 
thereof, which data stream is encoded according to 
the process of FIG. 53; 

FIG. 58 explains a manner of recording sub-picture 
(PXD) when the cfsplay mode of sub-picture is an 
interlaced mode; and 

FIG. 59 shows an example of the display control 
sequence table in the packet of FIG. 29. 

Encoding and decodng methods acconcfing to an 
embodiment of the present invention wOl be described 
below with reference to the accompanying drawings. In 
order to avoid a repetitive description, the same or sim- 
ilar reference numerate/symbols denote parts having 
the same or similar functions throughout the drawings. 

FIGS. 1 to 59 explain the image information encod- 
ing/decoding system according to an embodiment of the 
present invention. 

FIG. 1 schematically shows the structure of data 
recorded an double-sided optical disk OD as an infor- 
mation holding medium to which the present invention 
can be applied. 

Optical diskOD is a double-sided optical disk. Each 
surface of this optical oleic has a storage capacity of 
about 5 Gbytes. Many recording tracks are arranged 
between the lead-in area on the inner peripheral side of 
the cfsk and the lead-out area on the outer peripheral 
side of the cfsk. Each tack is constituted by many logi- 
cal sectors. Various pieces of information (compressed 
digital data) are stored in the respective sectors. 

FIG. 2 shows an example of the data structure of a 
picture fie (data for video) to be reootded in optical disk 
OD shown in FIG. 1. 

As shown in FIG. 2, this video file comprises fie 
management information 1 and picture (video) data 2. 
Video data 2 comprises video data blocks, audio data 
blocks, sub-picture data blocks, and DSI blocks in which 
pieces of i nformatio n (DSI; Disk Search In formati on) 
necessary to control data playback are respectively 
recorded. Each block is divided into packets having a 
predetermined data size for, e.g., each data type. The 



video data blocks, audio data blocks, and sub-picture 
data blocks are reproduced or played back in synchro- 
nism with the DSIs located before these blocks. 

More specifically, a system area in which system 
data used by disk OD is stored, a volume management 
information area, and a plurality of file areas are formed 
in the aggregate of logical sectors in FIG. 1. 

Oi the plurality of file areas, for example, fOe 1 corv 
tains main picture information ("VIDEO" in FIG. 2), sub- 
picture information ("SUB-PICTURE" In FIG. 2) having 
contents supplementary to the main picture information, 
audio in formation ("AUDIO" in FIG. 2}, playback infor- 
mation ("PLAYBACK INFO." in FIG. 2), and the Eke. 

FIG. 3 shows an example of the logical structure of 
a sub-picture infor ma t io n pack being subjected to the 
encoding (or the run-length compression). 

As shown at the upper portion of FIG. 3, one pack 
of the sub-picture information contained in the video 
data is constituted by, for example,, 2048 bytes (or 2k 
bytes). This one pack sub-picture information includes 
one or more sub-picture packets subsequent to the 
header of the leading pack. The first sub-picture packet 
is provided with run-length-compressed sub-picture 
data (SP DATA1) after its packet header. Sinrilarty, the 
second sub-picture packet is provided with mn-length- 
compressed sub-picture data (SP DATA2) after its 
packet header. 

Sub-picture unit header 31 is assigned to sub-pic- 
ture data unit 30 which is obtained by gathering a plural- 
ity of sub-picture data pieces (SP DATA1, SP DATA2, 
...). Here, the gathered sub-picture data pieces consti- 
tute one unit of the run-length compression. Following 
after sub-picture unit header 31 is pixel data 32 
obtained by run-length-comprassing the picture data of 
one unit which is, for example, image data on one hori- 
zontal (scanning) line of a two-dimensional display 
screen, and table 33 containing display control 
sequence ^formation of respective sub-picture packs. 

In other words, one unit of run-length-conpressed 
data 30 is formed of a group of the sub-picture data 
pieces (SP DATA1, SP DATA2, ...) of one or more sub- 
picture packets. In short, sub-picture data urtit 30 is con- 
stituted by sub-picture unit header SPUH 31, in which 
various parameters for sub-picture displaying are 
recorded, display data (compressed pixel data) PXD 32 
formed of run -length codes, and display control 
sequence table DCSQT 33. 

FIG. 4 shows part of the contents of sub-picture unit 
header 31 in one unit of run-length compressed data 30 
In FIG. 3 (cf. FIGL 31 for other part of SPUH 31). The 
data of a sub-picture (e.g., subtitles corresponding to a 
scene in a movie) to be recorded/transm i t te d (or com- 
municated] together with a main picture (e.g.. a picture 
of the movie) will be described betow. 

As win be seen from FIG. 4, reconded in sub-picture 
unit header SPUH 31 are: start address SPDDADR of 
sub-picture pixel data (display data); end address SPE- 
DADH of pixel data 32; data SPDSZ of the cfsplay-start 
position and display area (width and height) of pixel data 
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32 with respect to a TV display screen; and record start 
position SPDCSQTA of display control sequence table 

33 in the sub-picture data packet 

In a certain case, the following parameters may 
also be recorded in SPUH 31 : backrjound color SPCHI s 
designated by the system; sub-picture color SPCINFO 
designated by the system; pallet color number SPAD- 
JINFO of an errphasizing color designated by the sys- 
tem; modification information SPMOD of sub-picture 
pixel data 32; mixing ratio SPCONT of sub-picture (SP) to 
to main-picture (MP); start-timing (corresponding to the 
frame number of MP) SPDST of trie sub-picture; and 
start addresses SPLine 1 to SPUneN of decode data of 
respective lines. 

Incidentally, in a preferred embodiment of trie is 
present invention, start addresses SPLine 1 to SPLine 
N of decode data of the lines are respectively provided 
for a plurality of sub-picture fields. 

More specfficelly, as indicated by the lower portion 
in FIG. 4, various parameters (e.g., SPDDADR) having 20 
the following contents are recorded on sub-ptchre unit 
header 31: 

(1) the start address information (SPDDADR: an 
address relative to the beginning of the header) of ss 
the display data (subpicture pixel data) following 
the header; 

(2) the end address information (SPEDADR: an 
address relative to the beginning of the header) of 
the display data; so 

(3) information (SPDSZ) indicating the display start 
position and display range (width and height) of the 
display data on the monitor screen; and 

(4) record start position information (start address 
SPDCSQTA of the sub-picture display control as 
sequence table) of display control sequence table 

33 in the packet 

Fiffther, in a specific case of various embodi- 
ments according to the present invention, sub-pic- 
ture unit header SPUH 31 may include the following 40 
information pieces: 

(5) information (SPCHI) designated by the system 
and indicating the background color (a 16-color pal- 
ette number set by a story information table or a 
display control sequence table); 45 

(6) information designated by the system and indi- 
cating a sub-picture color (a 16-color palette 
number set by the story information table or the dis- 
play control sequence table); 

(7) information (SPAJDNFO) designated by the so 
system and indicating a sufanpicture emphasizing 
color (a color palette number set by the story infor- 
mation table or the cf splay control sequence table); 

(8) information (SPMOD) designated by the system 
and specifying the sub-picture image mode irtfbr- 55 
mation indicating either a field mode tor non-inter- 
laced display or a frame mode for interlaced display 
(when the pixel data to be compressed is d erf fried by 
various numb©' of bits, the number of bits of this 



pixel data can be specified by the content of this 
mode information); 

(9) information (SPCONT) designated by the sys- 
tem and indicating the mixing ratio of the sub-pic- 
ture to the main picture; 

(10) information (SPDST) indicating the display 
start timing of the sub-picture with the frame 
number (e.g., an I picture frame number of MPEG) 
of the main picture; 

(1 1) information (SPIine I to Spline N) indicating the 
start addresses (relative addresses with respect to 
the beginning of the sub-picture unit header) of the 
encoded data on the first to Nth lines of the sub-pic- 
ture; and 

Note that information SPCONT indicating the mix- 
ing ration of the sub-picture to the main picture repre- 
sents: the mixing ratio of the sub-picture with [system 
set value|/16 or [system set valuel/255, and the mixing 
ratio of the main picture with [16 - set vaiue]/16 or [255 - 
setvalue]/255. 

Sub-picture unit header 31 (or respective sub-pic- 
ture fields) includes the start address (SPLine 1 to 
SPLine N) of data to be decoded on every lines. For this 
reason, scrolling of only the sub-picture on the cfisplay 
screen can be realized by changing the designation of a 
decoding start line in accordance with an instruction 
from a microcomputer (MPU or CPU) on the decoding 
side. (How the scrolling is performed will be described 
later with reference to FIG. 21) 

In a certain embodiment of the invention, a 
field/frame mode (SPMOD) can be recorded on sub-pic- 
ture unit header 31 to indicate how the sii>-picture cor- 
responds to a TV field/frame of the NTSC schema 

In general, bit "0" is written in this field/frame mode 
recording portion (SPMOD). Upon reception of sub-pic- 
ture data unit 30, the decoding side determines from bit 
TJ" that the frame mode (non-interlaced mode) is set, 
and decodes the received encoded data in units of 
fines. As a result, a decoded image fike the one shown 
at the lower left position in FIG. 8 is output from the 
decoder. This image is displayed on the display screen 
of a monitor, a television (TV) set, or the Oka. 

If bit T is written in the fieldrtrame mode recording 
portion (SPMOD), the decoding side determines thai 
the field mode (interlaced mode) is seL In this case, 
after the encoded data is decoded in 1*1 its of Enes, the 
same data corresponding to two Dries are consecutively 
output as shown at the lower right position in FIG. 8. As 
a result, a frame corresponding to the TV interlaced 
mode can be obtained. With this operation, an image 
twice as large in amount as that in the frame mode (non- 
interlaced mode) can be displayed with the same data 
amount, although the image quality is kwrer than that in 
the frame mode. 

The data length (variable length) of the pixel data 
(run-length data) 32 of sub-picture shown in FIG. 3 or 4 
is determined depending on whether run-length com- 
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pression rules 1 to 6 in FIG. 5 or run-length compres- 
sion rules 11 to 15 in FIG. Bare used. 

Rules 1 to 6 in FIG. 5 are used when pixel data to 
be compressed has a multi-bit configuration (2 bits in 
this case). Rules 11 to 15 in FIG. 6 are used when pixel 
data to be compressed has a 1 -bit configuration. 

Whether run-length corrpresaon rules 1 to 6 or 11 
to 15 are to be used can be determined by the contents 
(ag., a bit width flag) of parameter SPMOD (see a por- 
tion near the middle of the table shown at the lower por- 
tion in FIG. 4) in sub-picture unit header 31. If, for 
example, the bit width flag of parameter SPMOD Is "1", 
the pixel data to be run-length compressed is 2-bit data, 
and hence rules 1 to 6 in FIG. 5 are used. If the bit width 
flag of parameter SPMOD is "0", the pixel data to be 
run-length compressed is 1 -bit data, and hence rules 11 
to 15 in FIG. 6 are used. 

Assume that four groups A, B, C, and D of compres- 
sion rules are provided respectively for four kinds of the 
bit-configuration of pixel data, when this pixel data can 
optionally have the bit-configuration off either of 1-bit 2- 
bft, 3-bit and 4-bit Under this assumption, when 
parameter SPMOD is constituted by 2-bit flag, the 1-bit 
pixel data using rule group A can be designated by the 
2-bit flag of "00". In similar manner, the 2-bit pixel data 
using rule group B can be designated by the 2-bit flag of 
"01", the 3-bit pixel data using rule group C can be des- 
ignated by the 2-bit flag off "10", and the 4-bit pixel data 
using rule group D can be designated by the 2-bit flag of 
"11". In this case, rules 11 to 15 of FIG. 6 can be used 
for the compression rules of group A, and rules 1 to 6 of 
FIG. 5 can be used for the compression rules of group 
B. When the contents of coding headers and the bit- 
configuration of pixel data, as well as the number of 
rules, are properly mooffied, the compression rules of 
groups C and D will be derived from rules 1 to 6 of FIG. 
5. 

FIG. 5 is a view for explaining run-length compres- 
sion rules 1 to 6 which are used in an encoding method 
according to an embodiment of the present invention, 
wherein pixel data constituting sub-picture pixel data 
(run-length data) 32 in FIG. 4 consists of a pluraity of 
bhs (2 bits in this case). 

FIG. 9 is a view for explaining compression odes 1 
to 6, in detail, in a case whereto pixel data constituting 
sii>-picture pixel data (run-length data) 32 in FIG. 4 con- 
sists of 2 bits. 

According to rule 1 on the first row in FIG. 5, when 

1 to 3 identical pixels continue, one encoded (run-length 
compressed) data unft is constituted by 4 bits. In this 
case, the first 2 bits represent the number of pixels fol- 
lowed, and the next 2 bits represent pixel data (pixel 
color information or the like). 

For exarrple, first compression data unit CU01 of 
picture data PXD before compressed, which is indicated 
by the upper portion in FIG. 9, contains 2 2-brt pixel data 
dO, d1 a (0000)b (b indicates binary data). In this case, 

2 identical 2-blt pixel data (00)b continua 



In this case, as indicated by the lower portion in 
FIG. 9, 2-bit display (10)b representing the number of 
pixels followed "2" is coupled to contents (00)b off the 
pixel data to form dO, d1 = (1000}b, which is data unit 

$ CU01 * of picture data PXD after compressed. 

In other words, (0000)b of data init CU01 is con- 
verted into (10D00)b of data unit CU01* according to 
rule 1. In this case, bit length compression is not practi- 
cally realized. However, for ecample, 3 continuous iden- 

w tical pixels (00)b, Le., CU01 = (OOOOOOJb, are 
compressed into CU01* = (1100)b. That is, the pixel 
data can be compressed by 2 bits. 

According to rude 2 on the second row in FIG. 5, 
when 4 to 15 identical pixels continue, one encoded 

75 data unit is constituted by 8 bits. In this case, the first 2 
bits represent a coding header indicating that encocfing 
is performed according to rule 2, the subsequent 4 bits 
represent the number of pixels followed, and the next 2 
bits represent pixel data. 

so For exarrple, second compression data unit CU02 
of picture data PXD before compressed, which is indi- 
cated by the upper portion in FIG. 9, contains 5 2-brt 
pixel data d2, d3, d4, d5, d6 = (0101010101)b. In this 
case, 5 identical 2-bit pixel data (01)b continua 

25 In this case, as indicated by the lower portion in 
FIG. 9, coding header (00)b, 4-bH display (01 01)b repre- 
senting the number of pixels followed "5", and contents 
(01 )b of the pixel data are coupled to each other to form 
d2, d6 = (00010101)b, which is data inft CU02* of pic- 

so ture data PXD after conrpressed. 

In other words, (0101010101)b (10-bit length) of 
data unit CU02 is converted into (000101 01)b (8-bit 
length) of data unit CU02* according to rule 2. In this 
case, the 10-bit data is compressed into the 8-bit data, 

35 i.e., the substantial bit length compression amount cor- 
responds to only 2 bits. If, however, the number of pixels 
followed is 15 (which corresponds to a 30 -bit length 
because 15 "01 "s of CU02 continue), the data is com- 
pressed into 8-bit data (CU02* = 00111101). That is, 

40 30-bit data can be compressed by 22 bits. The bit com- 
pressing effect based on rule 2 is therefore larger than 
that based on rule 1. In order to cope with run-length 
compression off a fine image with high resolution, rule 1 
is also required. 

45 According to rule 3 on the third row in FIG. 5, when 
1 6 to 63 identical pixels continue, one encoded data unit 
is constituted by 12 bits. In this case, the first 4 bits rep- 
resent a coding header indicating that encoding is per- 
formed according to rule 3, the subsequent 6 bits 

so represent the number of pixels followed, and the next 2 
bits represent pixel data. 

For exarrple, third compression data unit CU03 of 
picture data PXD before compressed, which is indicated 
by the upper portion in FIG. 9, contains 16 2-bit pixel 

ss data d7 to d22 = (101010 1010)b, In this case, 16 
identical 2-bit pixel data (10)b continue. 

In this case, as indicated by the lower portion in 
FIG. 9 V coding header (OOOO)b, 6-bit cfsplay (010000)b 
representing the number of pixels followed "16", and 
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contents (10)b of the pixel data are coupled to each 
other to form d7 to d22 = (000001 00001 0)b, which is 
data unit 01103* of picture date PXD after compressed. 

In other words, (101010 - • • 1 01 0)b(32*it length) 
of data unit CU03 Is converted into (000001000010)b 5 
(12-bit length) of data unit CU03* according to ride 3. In 
this case, the 32 -bit data is compressed into the 12-bit 
data, Le., the substantial bit length compression amount 
corresponds to 20 bits. If, however, the number of pixels 
followed is 63 (which corresponds to a 126-bit length io 
because 63 "10°6 of CU03 continue), the data is com- 
pressed into 12-bit data (CU03* = 00001 1 1 1 11 1 0). That 
is, 126-bit data can be compressed by 114 bits. The bit 
compressing effect based on rule 3 is therefore larger 
than that based on rule 2. is 

According to rule 4 on the fourth row in FIG. 5, 
when 64 to 255 identical pixels continue, one encoded 
data unft is constituted by 16 bits. In this case, the first 6 
bits represent a coding header indicating that encoding 
is performed according to rule 4 P the subsequent 8 bits so 
represent the number of pixels followed, and the next 2 
bits represent pixel data. 

For exanple, fourth co mp r e ssion data unit CU04 of 
picture data PXD before compressed, which is indicated 
by the upper portion in FIG. 9, contains 69 2-bit pixel zs 
datad23tod91 =(111111 - - - 1111)tx In this case, 69 
identical 2-bit pixel data (1 1)b continue. 

In this case, as indicated by the lower portion in 
FIG. 9, coding header (000000)b, 6-bit display 
(001001 01)b representing the number of pixels followed so 
"69°, and contents (1 1)b of the pixel data are coupled to 
each other to form d23 to d91 - (000000001001 01 11)b, 
which is data unit CU04* of picture data PXD after com- 
pressed. 

In other words, (111111 -•• 1111)b (138-bit ss 
length) of data unit CU04 is converted into 
(00000000100101 11)b (16-bit length) of data unit 
CU04* according to rule 4. In this case, the 138-bit data 
is c ompr e ss ed into the 16-bit data, Le., the substantial 
bit length compression amount corresponds to 1 22 bits. 40 
If, however, the number of pixels followed is 255 (which 
corresponds to a 510-bit length because 255 "11 "te of 
CU04 continue), the data is compressed into 16-bit data 
(CU04* = 0000001111111111). "mat is, 510-bit data 
can be compressed by 494 bits. The bit compressing 4S 
effect based on rule 4 is therefore larger than that based 
on rule 3. 

Acconfrxj to rule 5 on the fifth row in FIG. 5, when 
identical pixels continue from a switching point of a data 
unit of encoding to the end of a Ene, one encoded data so 
unit is constituted by 16 bits. In this case, the first 14brts 
represent a coding header indicating that encoding is 
performed according to rule 5, and the next 2 bits repre- 
sent pixel data. 

For example, fourth c om pr ess ion data unit CU05 of ss 
picture data PXD before compressed, which is incfcated 
by the upper portion In FIG. 9, contains one or more 2- 
bft pixel data d92 to dn = (000000 • * * 0000)b. In this 
case, a finite number of identical 2-bit pixel data (00)b 



continue. According to nrie 5, however, the number of 
pixels followed maybe 1 or more. 

In this case, as incScated by the lower portion in 
FIG. 9, cocfing header (00000000000000)b is coupled 
contents (00)b of the pixel data to form d92 to dn = 
(0000000000000000)b, which is data unit CU05* of pic- 
ture data PXD after compressed 

In other words, (000000 - • - 0000)b (unspecified 
bit length) of data unit CU05 is converted Into 
(0000000000000000)b (16-bit length) of data unit 
CU05* according to rule 5. According to rule 5, if the 
number of pixels followed up to the end of a line is 1 6 or 
more, a compressing effect can be obtained. 

According to rule 6 on the sixth row in FIG. 5, if the 
length of 1-line c ompre s s ed data PXD is not an integer 
multiple of 8 bits (i.e., not byte-afigned) at the end of a 
pixel line on which data to be encoded are arranged, 4- 
bit dummy data is added to the 1-fine conrpressed data 
to make 1-fne compressed data PXD coincide with a 
byte unit (i.e., for byte-aligning). 

For exaiTfrie, the total bit I ength of data units CU01* 
to CU05* of picture data PXD after compressed, which 
is indicated by the lower portion in FIG 9, is always an 
integer multiple of 4 bits. However, this length is not 
always an integer multiple of 8 bits. 

If, for example, the total bit length of data uhhs 
CUOr to CU05* is 1020, and 4 bits are required for 
byte-aligning, 4-bit dummy data CU06* = (0000)b is 
added to the end of the 1020-bit data to output data 
units CU01 * to CU06* as byte-aligned 1 024-bit data. 

Note that 2-bit pixel data is not Gmfted to data repre- 
senting four types of pixel colors. For exanple, pixel 
data (00)b may represent a background pixel of a sub- 
picture; pixel data (01)b, a pattern pixel of the sub-pic- 
ture; pixel data (10)b, the first emphasizing pixel of the 
sub-picture, and pixel data (1 1)b, the second emphasiz- 
ing pixel of the sUb-picture. 

As the number of bits constituting pixel data 
becomes larger, more types of sub-picture pixels can be 
designated, if, for example, pixel data is constituted by 
3-bit data (00O)b to (1 1 1)b, a maximum of eight types of 
pixel colors + pixel types (emphasizing effects) can be 
designated in sub-picture data to be run-length 
encoded/decoded. 

FIG. 6 is a view for explaining run-length compres- 
sion rules 11 to 15 which are used In an encoding 
method according to another e mb o dim ent of the 
present invention wherein pixel data constituting sub- 
picture pixel data (run-length data) 32 in FIG. 4 consists 
of one bit 

According to rule 1 1 on the first row in FIG. 6, when 
1 to 7 identical pixels continue, one encoded (run-length 
compressed) data unit is constituted by 4 bits. In this 
case, the first 3 bits represent the number of pixels fol- 
lowed, and the next 1 bit represents pixel data (informa- 
tion such as a pixel type). If, for example, 1 -bit pixel data 
TT, it Indicates a background pixel of a subpfoture. H 
this data is "1". it indicates a pattern pixel of the sub-pic- 
ture. 
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According to rule 12 on the second row in FIG. 6, 
when 8 to 15 identical pixels continue, one enooded 
data unit is constituted by 8 bits. In this case, the first 3 
bits represent a coding header (e.g., 0000) indcating 
that encoding is based on ride 12, the subsequent 4 bits 
represent the number of pixels followed, and the next 1 
bit represents pixel data. 

According to rule 13 on the third row in FIG. 6, when 
16 to 127 identical pixels continue, one encoded data 
unit is constituted by 12 bits. In this case, the first 4 bits 
represent a coding header (e.g., 0000) incficating that 
encoding is based on rule 13, the subsequent 7 bits rep- 
resent the nuiriber of pixels followed, and the next 1 bit 
represents pixel data. 

According to rule 14 on the fourth row in FIG. 6, 
when identical pixels continue from a switching point of 
a data unit of encoding to the end of a line, one enooded 
data unit is constituted by 8 bits. In this case, the first 7 
bits represent a coding header (eg., 0000000} incficat- 
ing that encoding is performed according to rule 14, and 
the next 1 bit represents pixel data. 

According to rule 15 on the sixth row in FIG. 6, if the 
length of Mine compressed data PXD is not an integer 
multiple of 8 bits (lb*, not byte-aligned) at the end of a 
pixel One on which data to be encoded are arranged, 4- 
brt dummy data is added to the 1 -ine compressed data 
to make 1-line compressed data PXD coincide with a 
byte unit (i.e., for byte-aEgning). 

An image encoding method (an encoding method 
using rui-length coding) wiD be described in detail next 
with reference to FIG. 7. 

FIG. 7 shows a case wherein pixel data constituting 
sub-picture pixel data (run-length data) 32 in FIG. 4 is 
constituted by the first to ninth lines, 2-bit pixels (having 
a maximum of four types of contents) are arranged on 
each line, and character patterns "A" and "B" are 
expressed by the 2-bit pixels on the respective Ones. 
The manner of encoding (run-length compressing) the 
pixel data on each line will be described in detail below. 

As indicated by the upper portion in FIG. 7, an 
image as a source is constituted by three types (a max- 
imum of four types) of pixel data. That is, 2-bit image 
data (00)b represents the pixel color of the background 
of the sub-picture; 2-bit image data (01)b, the pixel color 
of characters "A" and IT in the sub-picture; and 2-bit 
image data (10)b, an emphasizing pixel color with 
respect to sub-picture characters "A" and "B". 

When the original image containing characters "A M 
and "B* is scanned by a scanner or the fike, these char- 
acter patterns are read, from the left to right of each 
scanning line, in units of pixels. The picture data read in 
this manner is input to an encoder (denoted by refer- 
ence numeral 200 in the embodiment in FIG. 10 which 
will be described later) for performing run-length com- 
pression based on the present invention. 

This encoder can be constituted by a microcom- 
puter (MPU or CPU) fn which software for executing 
run-length compression based on rules 1 to 6 described 
with reference to FIG. 5 runs. This encoder software wil 



be described later with reference to the flow charts in 
FIGS. 13 and 14. 

Encocfing processing of run-length confessing a 
sequential bit string of character pattern "A" and "B" 
s read in units of pixels wifl be described below. 

In the case shown in FIG. 7, a source image is 
assumed to have three color pixels. More specifically, in 
picture data (the sequential bit string of character pat- 
terns "A" and "B") to be encoded, backgroimd color 
io pixel a *"is represented by 2-bit pixel data (00)b, charac- 
ter color pixel is represented by 2-bit pixel data 
(D1)b, and emphasizing color pixel V is represented by 
2-bit pixel data (10)b The bit count (= 2) of pixel data 
(e.g., 00 or 01) is also called a pixel width. 
75 For the sake of simplicity; in the case shown in FIG. 
7, the display width of picture data (sub-picture data) to 
be encoded is set to be 16 pixels, and the number of 
scanning lines (display height) is set to be 9 lines. 

Image data (sub-picture data) obtained from a 
so scanner is temporarily converted into a run-length value 
by the microcomputer. 

Consider the first line at the lower portion in FIG. 7. 
Three continuous pixels • are converted into 
( - *3); subsequent 1 pixel "0", (0*1); subsequent 1 pixel 
2s -# n ( (#*1); subsequent 1 pixel "o", (o*1); subsequent 
continuous 3 pixels • • n , (-*3); subsequent 1 pixel 
"o", (o*1); subsequent continuous 4 pixels "####", (#M); 
subsequent 1 pixel "o". (o*1); and last 1 pixel " • ( • *1). 

As a result as indicated by the intermediate portion 
so in FIG. 7, the run-length data (before compressed!) on 
the first One becomes 

n -*3/o*l/ri/o*l/-*3fo*1/#*4/o*1/-*r. This data is 
constituted by a combination of image information such 
as a character color pixel, and the number of pixels tol- 
as lowed which represents a continuation count 

Similarly, the pixel data strings on the second to 
ninth fries indicated by the upper portion in FfG. 7 
become the rurHength data strings on the second to 
ninth lines indicated by the intermediate portion in FIG. 

40 7. 

Consider the data on the first line. Since 3 back- 
ground color pixels " • • •'continue from the start of the 
line, compression rule 1 in FIG. 5 is used. As a result, 
first three pixels • •", i.e., {**3) 9 on the first line are 
45 encoded into (1100), which is a corribination of 2 bits 
(1 1) representing "S w and (00) representing background 
color pixel 

Since the next data on the first line is 1 pixel "o", 
rule 1 is used. As a result, next pixel "o", i.e., (o*1), on 

so the first line is encoded into (01 1 0). which is a combina- 
tion of 2 bits (01) representing "1 " and (10) representing 
errphasizing color pixel "o". 

Since the next data is 1 pixel T, rule 1 is used. As 
a result, next pixel "#\ i.e, (#*1), on the firm line is 

es encoded into (0101), which is a combination of 2 bits 
(01) representing "1" and (01) representing character 
color pixel (the portions corresponcfing to pixels 
■###..." are enclosed with the broken flnes in the inter- 
mediate and lower portions in FIG. 7). 
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Similarly, (o*1) is encoded into (0110); [**3), 
(1100); and (o*1), (0110). 

Since the subsequent data on the first line are 4 pix- 
els "####", compression rule 2 in FIG. 5 is used. As a 
result pixels i.a. (#*4), on the first line are 6 

encoded into (00010001), which is a combination of 2- 
bft header (00) representing that rule 2 is used, 4 bite 
(0100) representing the number of pixels followed "4", 
and (01) representing character color pixel "IT (the por- 
tions corresponding to "IT are enclosed with the broken to 
lines in FIG. 7). 

Since the subsequent data on the first line is 1 pixel 
"o", rule 1 is used. As a result, pixel *o", i.e., (0*1), is 
encoded into (0110), which is a combination of 2 bits 
(01) representing "1" and (10) representing emphasiz- is 
ing color pixel "o". 

Since the last data on the first line is 1 arte 1 is 
used. As a result, pixel " i.e., (-*1), is encoded into 
(0100), which is a combination of 2 bits (01) represent- 
ing "1" and (00) representing background color pixel 20 

In the above manner, run-length data 
"•*3/o*1^*1/o*1/-*3/o' t 1/# ,r 4/b*1/-*r (before com- 
pressed) on the first line is run-length compressed into 
(1100) (0110) (0101) (0110) (1100) (0110) (00010001) 25 
(0110) (0100), thereby conrpletkig the encoding of the 
first line. 

In the same manner as descrfoed above, encoding 
proceeds up to the eighth Gne. All the data on the ninth 
line are identical background color pixels " • • • ... n . In so 
this case, compression rule 5 in FIG. 5 is used. As a 
result run-length data u • *16" (before compressed!) on 
the ninth line is encoded into 1 6-bit data 
(0000000000000000), which is a combination of 14-bit 
header (00000000000000) representing that identical as 
backgroimd color pixels * • ..." continue to the end of 
the One and 2-bit pixel data (00) representing back- 
ground color pixBl " - u . 

Note that encoding based on rule 5 is used when 
data to be compressed starts midway along a Gne and 40 
continues to the end of the line 

FIG. 10 is a block diagram for explaining the flew of 
processing from the step of mass-producing high-den- 
sity optical risks having image information (cf. 31 + 32 + 
33 in FIG. 3) encoded on the basis of the present inven- 4s 
tion to the step of playing bade the information on the 
user side, together with the flow of processing from the 
step of broadcast* ng/cable^istilbuting image informa- 
tion encoded on the basis of the present invention to the 
step of receiving/claying back the information on the so 
user/subscriber side. 

Assume that run-let^jth data before compressed 
like the one shown at the intermediate portion in FIG. 7 
is input to encoder 200 in FIG. 10. Encoder 200 per- 
forms run-length compression (encoding) of the Input bs 
data by software processing based on compression 
rules 1 to6in FK3L5. 

When data of a logic structure like the one shown in 
FIG. 2 is to be recorded on optical disk OD in FIG. 1, 



encoder 200 in FIG. 10 performs run-length compres- 
sion processing (encoding processing) with respect to 
sub-picture data in FIG 3. 

Various data required to complete optical dusk OD 
are also input to encoder 200 in FIG. 10. These data are 
compressed on the basis of, e.g., the standards ol 
MPEG (Motion Picture Expert Group), and the com- 
pressed digital data are sent to laser cutting machine 
202 or modiiator/transmftter 210. 

In laser cutting machine 202, the MPEG-com- 
pressed data from encoder 200 is recorded on a mother 
disk (not shown) to manufacture optical disk master 
204. 

In two-sided high-density optical cfisk production 
plant 206, the info rma t io n of master 204 is transferred 
onto, e.g., a laser beam reflecting film on a 0.6-mm thick 
polycarbonate base. Two large-capacity polycarbonate 
base on which different pieces of master information are 
respectively transferred are Joined to each other to form 
a 1.2 mm thick two-sided optical disk (or a twensided 
disk having one information read surface). 

Double-sided high-density optical disks OD mass- 
produced by plant 206 are distributed to various types of 
markets and to users. 

Distributed disk OD is played back by disk player 
300 of the user. Disk player 300 includes decoder 101 
for decoding data encoded by encoder 200 into original 
information. The information decoded by decoder 101 is 
sent to, ag.. a monitor TV of the user to be visualized. 
In this manner, the end user can enjoy the original pic- 
ture information from large-capacity disk OD. 

The compressed information sent from encoder 
200 to modulator/transmitter 210 is modulated in 
accordance with predetermined standards and trans- 
mitted. For example, the compressed picture informa- 
tion from encoder 200 is broadcasted through a satellite 
(212), through corresponding audio i nfo r matio n. Alter- 
natively, the conpressed picture information from 
encoder 200 is transmitted through a cable (212), 
together with corresponding audio information. 

The conpressed picture/audio information broad- 
casted or transmitted through the cable is received by 
recei ver/demod utatar 400 of a user or subscriber. 
Receiver/demodulator 400 includes decoder 101 for 
decoding the data encoded by encoder 200 into the 
original information. The Information decoded by 
decoder 101 is sent to, ag., the monitor TV of the user 
to be visualized. In this manner, the end user can enjoy 
the original pictire information from the co mp r es sed 
picture information broadcasted or transmitted through 
the cable. 

FIGL 11 isaUockcfiagramshowinganeni^ment 
(non-interlaced specaication) of decoder hardware for 
executing image decoding (run-length expansion) 
based on the present invention. Decoder 101 (see FIG. 
10} for decoefng run-length con pr e s sed 6ub-picture 
data SPD (corresponding to data 32 in FIG. 3) can have 
an arrangement like the one shown in FIG. 11. 
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A 6ub-picture data decoder for run-length expand- 
ing a signal containing run-length compressed pixel 
data having a format like the one shown in FIG. 4 will be 
described below with reference to FIG. 11. 

As shown "m FIG. 11, decoder 101 comprises data 5 
I/0 1 02 to which sub-picture data SPD is input; memory 
108 retaining sub-picture data SPD; memory control 
105 for controllng the read operation of memory 108; 
continuous code length detector 106 for delecting the 
continuous code length (coding header) of one unit (one w 
block) from the run information of encoded data (run- 
length compressed pixel data) read out from memory 
108, so as to output separation information for the con- 
tinuous code length; coding data separator 103 for 
extracting one-block encoded data in accordance with 75 
the information from continuous code length detector 
106; run-length setter 107 for receiving a signal output 
from coding data separator 103 and representing the 
run information of one compression unit, together with a 
signal (period signal) output from continuous code 20 
length detector 1 06 and indicating the number of contin- 
uous bits "01s, i.e., the nurri>er of data bits "0"fe continu- 
ing from the 6tart of the coding data of one block, and 
calculating the nunrfcer of pixels followed in one block 
from these signals; pix. color out-stage 104 (Fast- 25 
in/Fast-out type) for receiving pixel color information 
from coding data separator 103 and the period signal 
output from run -length setter 107 and outputting the 
color information only for the corresponding interval; 
microcomputer 1 12 for loading the header data (see so 
FIG. 4) in sii>picture date SPD read out from memory 
108, and performing various types of processing set- 
tings and control operations on the basis of the loaded 
data; adcfress control 109 for controlling the read/write 
addresses of memory 108; insufficient pix. color setter ss 
111 in which color information corresponding to a fine 
on which no run information is present is set by micro- 
computer 1 12; and display activator 1 1 0 for determining 
a display area, of a TV screen or the fike, in which a sub- 
picture is to be cfisplayed. 40 

Incidentally, system timer 120 and buffer memory 
121 are coupled to MPU 112 of decoder 101. which wffl 
be referred to in the explanations of FIGS. 53 to 57. 

In other words, as descrtoed in FIG. 1 1 , run-length 
compressed sub-picture data SPD Is sent to the internal as 
bus of decoder 101 through data VO 102. Sub-picture 
data SPD sent to the bus is supplied to memory 108 
through memory control 105 to be stored therein. The 
internal bus of decoder 101 is connected to cocEng data 
separator 103, continuous code length detector 106, so 
and microcomputer (MPU or CPU) 112. 

Sub-picture unit header 31 of the sub-picture data 
obtained from memory 108 is read by microcomputer 
112. Microcomputer 112 detects the various parame- 
ters shown in FIG. 4 from read sub-picture unit header ss 
31. In accordance with the detected parameters, the 
start address of decoding (SPDDADR) is set at address 
controller 109; cfcplay activator 110 receives informa- 
tion (SPDSZ) indicating the display start position, dis- 



play width, and tfsplay height of the sub-picture; and 
coding data separator 1 03 receives display width (or the 
number of dots of the brie) of the sub-pictura The 
parameters detected by microcomputer 112 are then 
stored in the internal registers of respective circuit 
blocks (109, 110, and 103). Thereafter, microconrputer 
112 can access the parameters stored in the circuit 
blocks (109, 110, and 103). 

Address controller 109 accesses memory 108 
through memory controller 1 05 in response to the starl- 
ing address (SPDDADR) of decodng set at the register 
of controller 109, so that the reading of the sub-picture 
data to be decoded starts. The sub-picture data read 
from memory 108 is suppBed to coding data separator 
1 03 and to continuous code length detector 1 06. 

The coding header (2 to 14 bits according to rules 2 
to 5 In FIG. 5) of run-length compressed sub-picture 
data SPD is detected by continuous code length detec- 
tor 10a The number of continuous pixels of identical 
pixel data in sub-picture data SPD is detected by run- 
length setter 107 based on a signal from continuous 
code length detector 1 06. 

More specifically, continuous code length detector 
106 counts the number of bits of D" in the data read 
from memory 108 in order to detect the coding header 
(cf. FIG. 5). In accordance with the value or content of 
the detected coding header, detector 106 supplies sep- 
aration information SEP. INFO, to coding data separator 
103. 

In response to separation information SEP.INFO., 
coding data separator 103 sets the continuous pixel 
numbers (run information) at run length setter 107. At 
the same time, separator 1 03 sets the pixel data (sepa- 
rated data indicating the pixel color) at pixel color output 
stage 104 of a FIFO (fir&MrWf irst-out) type. In this case, 
coding data separator 103 counts the pixel number cf 
the sub-picture data, and compares the counted pixel 
number with the display width (or the number of pixels 
on one Ene) of the sub-picture. 

Here, if no byte-aligning is established when the 
decoding of one One is completed (or if the bit length of 
one line data is not the integral of the unit of number 
"8"), coding data separator 103 discards or neglects the 
trailing 4-bit data on the Ene, regarding as the dummy 
data added at the time of encoding. 

Run-length setter 107 supplies pixel color output 
stage 104 with a PERIOD SIGNAL for outputting the 
pixel data, based on the continuous pixel number (run 
infor m ation), pixel dot clock DOTCLK, and horizon- 
taWertical sync signals H-SYNG/V-SYNC. Pixel odor 
output stage 104 outputs the pixel data from coding 
data separator 103, as the decoded display data, during 
the active period of the pixel data output signal 
(PERIOD SIGNAL), or during the period of outputting 
the same pixel color. 

Within the above-mentioned active period of the 
PERIOD SIGNAL if the start line of decoding is 
changed by the instruction from microcomputer 112, a 
certain iine(s) having no run information could exists. 
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When the no-run -information line(s) is(are) found during 
the decoding, insufficient pixel color setter 111 sends 
data (COLOR INFO.) of preset insufficient pixel color to 
pixel color output stage 104. Then, pixel color output 
stage 1 04 outputs the data (COLOR INFO.) of the insuf- 6 
f icierrt pixel color from insufficient pixel color setter 111, 
so long as the no-rurHnformation line(s) is(are) input to 
coding data separator 103. 

More specifically, in the case of decoder 1 01 in FIG. 
11, if input sub-picture data SPD includes no image 
data, microco mp uter 1 12 sets pixel color information as 
shortage data in insufficient pix. color setter 111. 

A display enable signal for determining a specific 
position on a monitor screen (not shown) at which a 
decoded sub-picture is to be displayed is supplied from 
display activator 110 to pix color out-stage 104 in syn- 
chronism with a horizontal/vertical sync signal for a sub- 
picture image. Display activator 110 sends a color 
switching signal to pix. color out-stage 104 in accord- 
ance with a color information instruction from microcom- 
puter 112. 

After the processing settings of microcomputer 1 1 2, 
address control 109 sends address data and various 
timing signals to memory control 105, continuous code 
length detector 106, coding data separator 103, and 
run-length setter 107. 

When a pack of sub-picture data SPD is received 
through data I/O 102 and stored in memory 108, micro- 
computer 112 reads the contents of the pack header of 
data SPD (e.g., a start address of decoding, an end 
address of decoding, a display start position, a display 
width, and a cfisplay height}. Microcomputer 112 sets a 
start address off decoding, an end address of decoding, 
a display start position, a display width, a display height, 
and the like in display activator 1 1 0 on the basis of the 
read contents. At this time, a specific bit configuration of 
comp r essed pixel data (2-bit pixel data in this case) can 
be determined by the contents of sub-picture unit 
header 31 in FIG. 4. 

The operation of decoder 101 in a case wherein 
compressed pixel data has a 2 -bit configuration (rules 1 
to 6 in FIG. 5 are used) will be described 

When the start address decoding is set by micro- 
computer 112, address control 109 sends correspond- 
ing address data to memory control 105, and a read 
start signal to continuous code length detector 106. 

In response to the sent read start signal, continu- 
ous code length detector 106 sends a read signal to 
memory control 105 to load encoded data (compressed 
sub-picture data 32). Continuous coda length detector 
106 then checks whether all the upper 2 bits of the read 
data are TTa 

If all the Wis are not "O^s, it is determined that the 
block length of the unit of compression is 4 bits (see rule 
1 1n FIG. 5). ss 

tf the bits (upper 2 bits) are "0*s, the subsequent 2 
bits (upper 4 bits) are checked. If they are not "0 n s, ft is 
determined that the block length of the unit of compres- 
sion is 8 bits (see rule 2 in FIG. 5). 



If the bits (upper 4 bits) are "O's, the subsequent 2 
bits (upper 6 bits) are checked. If they are not "©"s, it is 
determined that the block length of the unit of compres- 
sion is 12 bits (see rule 3 in FIG. 5). 

If the bits (upper 6 bits) are "0's. the subsequent 8 
bits (upper 14 bits) are further checked. If they are not 
"0"s. it is determined that the block length of the unit of 
compression is 16 bits (see rule 4 in FIG. 5). 

If the bits (upper 14 bits) are "0"s, it is determined 
that the block length of the unit of compression is 16 
bits, and identical pixel data continue up to the end of 
the line (see rule 5 in FIG. 5). 

If the number of bits of the pixel data read up to the 
end of the line is an integer multiple of 8, the pixel data 
is used as it is. If the number of bits is not an integer 
multiple of 8, it is determined that 4-bit dummy data is 
required at the end of the read data to realize byte^al idl- 
ing (see rule 6 in FIG. 5). 

Coding data separator 103 extracts 1 -block data 
(the unit of concession) of sub-picture data 32 from 
memory 108 an the basis of the above determination 
result obtained by continuous code length detector 106. 
Coding data separator 103 separates the extracted 1- 
block data into the number of pixels followed and pixel 
data (e.g., pixel color information). The separated data 
of the number of pixels followed (RUN INFO.) is sent to 
run-length setter 107, and the separated pixel data 
(SEPARATED DATA) is sent to pix. color out-stage 1 04. 

Display activator 110 generates a display enable 
6ignal (enable signal) for designating a sub-picture cfis- 
ptay interval in synchronism with a pixel dot clock 
(PIXEL-DOT CLK), a horizontal sync signal (H-SYNC), 
and a vertical sync signal (V-SYNC) in accordance with 
the display start position information, the display width 
information, and the display height information received 
from microcomputer 112. This display enable signal is 
output to run-length setter 107. 

Run-length setter 107 receives a signal output from 
continuous code length detector 106 and indicating 
whether the current block data continues to the line end, 
and continuous pixel data (RUN INFO.) from coding 
data separator 1 03. On the basis of the signal from con- 
tinuous code length detector 106 and the data from cod- 
ing data separator 103, run-length setter 107 
determines the number of pixel dots of a block which is 
being decoded, and outputs a display enable signal 
(output enable signal) to pix. color out-stage 1 04 during 
an interval corresponding to the number of dots. 

Pix. color out-stage 104 is enabled during an inter- 
val in which an period signal is received from run-length 
setter 107. In this interval, pix. color out-stage 104 
sends the pixel color information received from coding 
data separator 103, as decoded display data, to a dis- 
play unit (not shown) in synchronism with the pixel dot 
dock (PIXEL- DOT CLK). That is, pix. color out-stage 
104 outputs the same display data corresponding to the 
pixel pattern continuous dots of the block which is being 
decoded. 
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Upon determining that the encoded data are identi- 
cal pixel color data continuing to the fine end, continu- 
ous code length detector 106 outputs a signal for a 
continuous code length of 16 bits to coding data separa- 
tor 103, and also output to run-length setter 107, a sig- 5 
nal indicating that identical pixel data continue up to the 
line end. 

Upon reception of the above signal from continuous 
code length detector 106, run-length setter 107 outputs 
an output enable signal (period signal) to pa. color out- 10 
stage 104 to keep the color information of the encoded 
data in an enable state unti horizontal sync signal H- 
SYNC is inactivated. 

When microcomputer 112 changes the start fme of 
decocfing to scroll the displayed contents of a sub-pic- 15 
ture, no data line used for decoding may be present in a 
preset display area (i.e., a decoding line shortage may 
occur). 

In decoder 101 in FIG. 11, in order to cope with 
such a case, pixel color data for compensating tor a line so 
shortage is prepared in advance. When a One shortage 
is actually detected, the current display mode is 
switched to an insufficient pixel color data display mode. 
More specifically, when a data end signal is supplied 
from address control 109 to display activator 1 10, die- ss 
play activator 110 sends a color switching signal 
(COLOR SW SIGNAL) to pix. color out-stage 104. In 
response to this switching signal, pix. color out-stage 
1 04 switches the mode of outpiMting decoded pixel color 
data from the encoded data to the mode of outputting so 
decoded color information (COLOR INFO.) from display 
activator 110. This switched state is kept during an 
insufficient Gne display interval (DISPLAY ENABLE = 
active). 

When the above line shortage occurs, a decoding ss 
operation may be stopped instead of using insufficient 
pixel color data. 

More specifically; when, for example, a data end 
signal is input from address control 109 to display acti- 
vator 1 10, a color switching signal tor designating a dis- 40 
play slop may be output from display activator 110 to 
pix. color out-stage 1 04. Pix color out-stage 104 keeps 
stopping the display of a sub-picture during an interval 
in which this display stop designation color switching 
signal is active. 45 

FIG. 8 shows two display modes (non-interlaced 
display and interlaced display) to explain how character 
pattern "A" of the pixel data (six>pictLffe data) encoded 
in FIG. 7 is decoded. 

Decoder 101 in FIG. 11 can be used to decode so 
compressed data Kfce the one shown at the upper por- 
tion in FIG. 8 into interlaced display data like the one 
shown at the lower left portion in FIG. 6. 

In co ntras t to this, when compressed data Dee the 
one shown at the upper portion In FIG. 8 is to be ss 
decoded into interlaced display data shown at the lower 
right portion in FIG. 8, a Ihe doubier for scanntog the 
same pixel Gne twice (ag., re-scanning Dne #10, in an 
even field, which has the same contents as those of line 



#1 in an odd field; switching in units of V-SYNC pulses) 
is required. 

When an image display amount equivalent to that in 
the interlaced display mode is to be displayed in the 
non-interlaced display mode, another line dottier (ag., 
line #1 0 having the same contents as those of line #1 at 
the lower end portion in FIG. 8 is made to follow fine #1 ; 
switching in units of H-SYNC pulses). 

FIG. 12 is a block diagram for explaining an embod- 
iment (interlaced specifications) of decoder hardware 
having the above fine doubier function. Decoder 101 in 
FIG. 10 may be constituted by a decoder having the 
arrangement shown in FIG. 12. 

In the arrangement in FIG. 12, microcomputer 112 
detects the generation timings of odd and even fields in 
the interlaced display mode on the basis of a horizon- 
tal/vertical sync signal for a sub-picture. 

Upon detection of an odd field, microcomputer 1 12 
supplies, to select signal generator 118, a mode signal 
indicating that the current field is an odd field. As a 
result, select signal generator 118 outputs a signal to 
selector 1 15 to select decoded data from decoder 101. 
Decoder 101 then outputs the pixel data (see the lower 
right portion in FIG. 8) of lines #1 to #9 in the odd field, 
as an video output, to an external unit through selector 
1 15. In this case, the pixel data of ines #1 to #9 in the 
odd field are temporarily stored in line memory 114. 

Upon detecting that the odd field has shifted to an 
even field, microcomputer 1 1 2 suppfies, to select signal 
generator 1 18, a mode signal incficating that the current 
field is an even field. As a result, select signal generator 
118 outputs a signal to selector 115 to select the data 
stored in Gne memory 114. Line memory 114 then out- 
puts the pixel data (see the lower right portion in FIG. 8) 
of lines #1 0 to #1 8 in the even field, as a video output, to 
the external unit through selector 1 15. 

In this manner, the sub-picture image (character "A" 
in FIG. 6) of lines #1 to #9 in the odd field is synthesized 
with the sub-picture image (character n A w in FIG. 8) of 
lines 1 0# to #18 in the even field, thereby reafizing inter- 
laced display. 

Note that sub-picture unit header 31 of the sub-pic- 
ture data in FIG. 4 includes a parameter bit (SPMOD) 
indicating a frame olsplay mode/field display mode tor a 
TV screen. 

For example, an image display amount equivalent 
to that in the interlaced display mode is displayed in the 
non-interlaced display mode In the following manner. 

Upon loading sub-picture unit header 31, micro- 
computer 112 in FIG. 12 can determine from the set 
value of parameter SPMOD (active = "1 inactive = D") 
whether the interlaced mode (active "1") or the non- 
interlaced mode (inactive TH is set 

In the arrangement in FIG. 12, H parameter SPMOD 
Is active = T microcomputer 1 12 detects that the Inter- 
laced mode is set, and sends a mode signal incficating 
the interlaced mode to select signal generator 118. 
Select signal generator 1 18 sixties a switching signal 
to selector 115 every time horizontal sync signal H- 
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SYNC is generated. Selector 115 alternately switches 
the decoded output (decoded data) of the current field 
from decoder 101 and the decoded output of the current 
field temporarily stored in line memory 114 every time 
horizontal sync signal H-SYNC is generated, and out- $ 
puts a video output to an external TV set or the ike. 

Whan the current decoded data and the decoded 
data in line memory 1 14 are switched for each H-SYNC, 
a picture having a density (the number of horizontal 
scanning fines) twice that of the original image io 
(decoded data) is displayed on the TV screen in the 
interlaced mode. 

In decoder 101 having the above arrangement, 
sequentially input bit data are read by 2 to 1 6 bits while 
being counted bit by bit from the beginning of a decoded 75 
data unit block, and are decoded, instead of being 
decoded after read by one line. In this case, the bit 
length (4 bits, S bits, 12 bits, 16 bits, or the Gke) of one 
decoded data unit is detected immediately before a 
decoding operation. For example, compressed pixel 20 
data is decoded (played back) into three types of pixels 
("• "o". and in FIG. 7) in real time in the unit of the 
detected data length. 

tn decocfing pixel data encoded according to rules 1 
to 6 in FIG. 5, decoder 101 may have a bit counter and 2s 
a data buffer (line memory 1 14 or the like) having a rel- 
atively small capacity. In other words, the circuit 
arrangement of decoder 101 can be relatively simpli- 
fied, and the overall apparatus including this encoder 
can be reduced in size. *> 

That is, the encoder of the present invention 
requires no large code table in the coder unlike in the 
conventional MH coding method, and need not read 1tie 
same data twice in an encoding operation unlike in the 
arithmetic coding method. In addition, the decoder of ss 
the present invention requires no relatively complicated 
hardware like a multiplier, and can be realized by adding 
simple circuits such as a counter and a small -capacity 
buffer. 

According to the present invention, run-length com- 40 
pressbn/encoding and run-length expansion/decoding 
of many types of pixel data (a maximum of four types of 
pixel data each having a 2-bit configuration) can be real- 
ized with a relatively simple arrangement. 

FIG. 13 is a flow chart for executing image encoding 4s 
(run-length compression) according to an enixxfrnent 
of the present invention and for explaining software exe- 
cuted by the encoder (200) in FIG. 10. 

A series of encoding operations based on rules 1 to 
6 in FIG. 5 is executed, as software processing, by the so 
microcomputer in encoder 200 in FIG. 10. The overall 
encoding processing can be performed by encoder 200 
in accordance with the flow chart in FIG. 13. Run-length 
compression of image data in sub-picture data can be 
performed in accordance with the flow chart in FtG. 1 4. fis 
(Note that the explanation for encoding display control 
sequence table DCSQT 33 of FIG. 3 is omitted hera 
The encoding of DCSQT 33 will be explained later with 
reference to FIG. 53.) 



In this case, when the number of lines and the 
number of dots of image data are designated by a Key 
input operation (step ST801). the corrputer In the 
encoder 200 prepares a header area tor sub-picture 
data, and initializes the line count to "0" (step STB02). 

When an image pattern is sequentially input in units 
of bits, the computer in the encoder 200 obtains the first 
1-pixel data (2 bits in this case) t and stores the pixel 
data. In addition, the computer initializes the pixel count 
to T. and the dot count to "1" (step ST803). 

Subsequently, the computer in encoder 200 obtains 
the next pixel data (2 bits) of the pixel pattern, and com- 
pares it with the stored preceding image data (step 
STB04). 

If it is determined from the conrparison result thai 
the pixel data do not match each other (NO in step 
STB05), encoding 1 is performed (step ST8Q6), and the 
current pixel data is stored (step STBQ7). The pixel 
count is then incremented by one, and the dot count is 
also incremented by one accordingly (step ST808). 

If it is determined from the comparison result thai 
the pixel data match each other (YES in step STB05). 
encoding 1 in step ST806 is skipped, and the flow 
advances to step ST803. 

After the pixel count and the dot count are incre- 
mented (step ST808). the computer in encoder 200 
checks whether a pixel fine which is currently encoded 
is the end of a pixel fine (step 809). If the pixel line is the 
end of the line (YES in step ST809). encoding 2 is per- 
formed (step ST810). If the pixel fine is not the end of 
the line (NO in step STB09), the flow returns to step 
ST840, and the processing in steps ST804 to ST808 is 
repeated. 

Upon conrpletion of encoding 2 in step ST810, the 
computer in encoder 200 checks whether a bit string 
after encoding is an integer multiple of 8 bits (byte- 
aligned) (step ST811A). If the bit string is not byte- 
aligned (NO in step STB11A). 4-bit dummy data (0000) 
is added to the end of the bit string after encoding (step 
STB1 1 B). After this dummy addition processing, or if the 
bit string after encoding is byte-aligned (YES in step 
STB11A), the line counter of the corrputer in the 
encoder (e.g., the general-purpose register in the micro- 
computer} is incremented by one(step ST812). 

If the current line is not the final line (NO Instep 
ST813) after the line counter is incremented, the flow 
returns to step ST603, and the processing in steps 
STB03 to ST812 is repeated. 

If the current line is the final line (YES in step 
STB 13) after the line counter is incremented, the encod- 
ing processing (run-length compression of the bit string 
of the 2-bit pixel data) is completed. 

FIG. 14 is a fbw (tail for explaining the contents of 
encocfing 1 in FIG. 13. 

In encoding 1 (step ST806) In FIG. 13, pixel data to 
be encoded is assumed to have a 2-bit width, and 
hence run-length compression rules 1 to 6 In FIG. 5 are 
used. 
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In correspondence with rules 1 to 6, computer soft- 
ware is executed to determine whether the pixel count is 
0 (step ST901). 1 to 3 (step ST902), 4 to 15 (step 
ST903), 16 to 63 (step ST904), or 64 to 255 (step 
ST905), or indicates the end of a line (step ST906), or is $ 
256 or more (step ST907). 

The computer in encoder 200 determines the 
number of run field bits (one unit length of identical pixel 
data) on the basis of the above determination result 
(steps ST908 to ST913). and ensure an area cone- to 
sponcfing to the determined number of run field bits after 
sii>pictajre unit header 31. The number of pixels fol- 
lowed is output to the run field ensured in this manner, 
and the pixBl data is output to the pixel field. These data 
are recorded in a memory unit (not shown) in encoder is 
200 (step ST914). 

FIG. 1 5 is a flow chart for executing image decoding 
(run-length expansion) according to an embodiment of 
the present invention, and for explaining software exe- 
cuted by microcomputer 1 12 in FIG. 11 or 12. (Note that 20 
the explanation for decoding display control sequence 
table DCSQT 33 of FIG. 3 is omitted here. The decoding 
of DCSQT 33 will be explained later with reference to 
FIGS. 54 to 57.) 

FIG. 1 6 is a flow chart for explaining the contents of 25 
the decoding step (step ST1 005) used in the software in 
FIG. 15. 

Microcomputer 112 loads first header 31 of run- 
length compressed sub-picture data (pixel data has a 2- 
brt configuration), and analyzes its contents (see FIG. so 
4). The data is then decoded on the baste of the ana- 
lyzed contents of the header. The number of Ones and 
the number of dots of the data are designated. When 
the number of lines and thB number of dots are desig- 
nated (step ST1001), the line count and the dot count ss 
are initialized to "0" (steps ST1002 and ST1003). 

Microcomputer 112 sequentially receives the data 
bh string following sub-picture unit header 31. and 
counts the number of dots and the dot count Microcom- 
puter 112 then subtracts the dot count from the number 40 
of dots to obtain the number of pixels followed (step 
ST1004). 

Upon calculating the number of pixels followed in 
this manner, microcomputer 1 12 executes decoding in 
accordance with the number of pixels followed (step 4S 
ST 1005). 

After the decodng processing in step ST1005, 
microcomputer 112 adds the dot count to the number of 
pixels followed to obtain a new dot count (step ST1006). 

Microcomputer 112 sequential toads data and so 
executes the decoding processing in step ST1005. 
When the accumulated dot count coincides with the ini- 
tially sat line end count (the position of the end of the 
line), microc ompu ter 1 12 terminates decoding process- 
ing for 1-fine data (YES in step ST1007). ss 

If the decoded data is byte-afigned (YES in step 
ST1008A), dummy data is removed (step ST1008B). 
The line count is then incremented by one (step 
ST1009). The processing in steps ST1002 to ST1009 is 



repeated until the final fine (NO in step ST1010). If the 
current line is the final fine (YES in step ST1010), the 
decoding processing is terminated. 

For exanple, FIG. 16 shows the contents of the 
decoding processing in step ST1005 in FIG. 15. 

From the beginning of this processing, 2 bits are 
obtained, and it is checked whether the b'rts are "0". This 
determination 6tep is repeated (steps ST1101 to 
ST1 1 09). With this processmg, the nunfcer of pixels fol- 
lowed, i.e., the number of continuous runs, correspond- 
ing to or -length compression rides 1 to 6 is determined 
(steps ST1 110 to ST1 113). 

After the number of continuous runs is determined, 
the subsequent 2 bits obtained afterward is used as a 
pixel pattern (pixel data; pixel color information) (step 
ST1114). 

When the pixel data (pixel color information) is 
determined, index parameter T is set to 0 (step 
ST1 1 15). Until parameter T coincides with the number 
of continuous runs (step ST1 1 1 6), a 2-bit pixel pattern is 
output (step ST1 117). and parameter V is incremented 
by one (step ST1 1 18). After identical data correspond- 
ing to one unit is output, the decoding processing is 
completed. 

As described above, according to this sub-picture 
decoding method, sub-picture data can be decoded by 
simple processing, i.e., only determination processing 
tor several bits, data block separation processing, and 
data bit counting processing. For this reason, a large 
code table used in the conventional MH coding method 
or the like is not required, and the processing/arrang e- 
ment for decoding encoded bit data into original pixel 
information can be simplified. 

In the above embedment the encoded bit length of 
identical pixels corresponding to one unit can be deter- 
mined by reading a maximum of 1 6-bit data in a decod- 
ing operation. However, the encoded bit length is not 
limited to this value. For example, the encoded bit length 
may be 32 or 64 bits. However, as the bit length 
increases, a data buffer having a larger capacity is 
req tired. 

In addition, in the above em bo di m en t, pixel data 
(pixBl color information) includes pieces of color infor- 
mation of three colors selected from, e.g., a 16-color 
color palette Instead of usrig such pixel data, pieces of 
amplitude information of the three primary colors (eg., 
red R, green Q, and blue B; or intensity Y, chroma-red 
Cr, and chroma-blue Cb) can be expressed by 2-bit 
pixel data. That is. pixel data is not limited to specific 
color Information. 

FIG. 17 shows a modification of the embodiment of 
Fia 11. In the embodiment of FIG. 11, the header is 
separated by microcomputer 1 12 In software basis. On 
the other hand, in the embodiment of FIG. 17, the 
header is separated based on the hardware in decoder 
101. 

More specifically, as shown in FIG. 17, sub-picture 
data SPD subjected to the run-length concession is 
sent, via data UO 102, to the internal bus of decoder 
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101 . Sub-picture data SPD on the internal bus is sent to 
memory 108 through memory controller 105, and is 
then stored in memory 1 08. Hie internal bus of decoder 
101 is connected to coding data separator 103, to con- 
tinuous code length detector 106, and to header sepa- 5 
rator 113 which is coupled to microcomputer (MPU or 
CPU) 112. 

Sub-picture unit header 31 of the sub-picture data 
read from memory 108 is read by header separator 113. 
Separator 113 detects various parameters shown in 10 
FIG. 4 from read header 31. Based on the detected 
parameters, separator 113 sets at address controller 
109 the start address (SPDDADR) of decocting; at dis- 
play activator 1 10 the ^formation (SPDSZ) of the dis- 
play width and display height of the sub-picture as well « 
as the cSsplay start position of the sub-picture; and at 
coding data separator 103 the display width (the 
number of dots of the line). These set information 
pieces are stored in the internal registers of correspond- 
ing circuit blocks (109, 1 10, and 103). Thereafter, micro- 20 
computer 112 can access the parameters stored in the 
circuit blocks (109. 1 10, and 103). 

Address controller 109 accesses memory 108 
through memory controller 105 in response to the start- 
ing address (SPDDADR) of decocfing set at the register 25 
of controller 109, so that the reading of the sub-picture 
data to be decoded starts. The sub-picture data read 
from memory 108 is suppOed to coding data separator 
103 and to continuous code length detector 106. 

The coding header (e.g., 2 10 14 bits of rules 2 to 5 so 
in FIG. 5) of run-length-corrpressed sii>picture data 
SPD is detected by continuous code length detector 
106. The number of pixels followed with respect to the 
same pixel data within sub-picture data SPD is detected 
by run-length setter 107 in accordance with the signal ss 
from continuous code length detector 1 06. 

Another decoding method differing from the decod- 
ing method of FIGS. 15 and 16 will be explained with 
reference to FIGS. 17to21. 

FIG. 16 is a flow chart for explaining the former 40 
processing of the image decoding (run-length expan- 
sion) according to another embedment of the present 
invention. 

When the decoding starts, respective blocks in 
decoder 1 01 of FIG. 1 7 are initialled (e.g., registers are 45 
cleared and counters are reset). Thereafter, sub-picture 
unit header 31 is read, and the contents (various param- 
eters shown in FIG. 4) of header 31 are set at internal 
registers of header separator 1 13 (step ST1200). 

After the parameters of header 31 are set at the bo 
registers of header separator 113, microcomputer 112 
is informed of the end status of reading of header 31 
(stepST1201). 

When microcomputer 112 receives the end status 
of header reading, It designates the start line of deood- ss 
ing (eg., SPLine 1 in FICL 4). The designated start line 
of decoding is sent to header separator 113 (step 
ST1202). 



When header separator 113 receives the desig- 
nated start line of decocfing, separator 1 13 refers to the 
various parameters set in its own registers. Then, 
according to the parameters set in the registers of sep- 
arator 113, the following data set operations are per- 
formed (step ST1 203): 

* the designated decoding start Ene address (SPD- 
DADR in FIG. 4) and the decoding end address 
(SPEDADR in FIG. 4; the address obtained by rela- 
tively-shifting by one line from the start line 
address) are set at address controler 109; 

* the display start position, cfisplay width, and display 
height of the decoded sub-picture (SPDSZ in FIG. 
4) are set at display activator 110; and 

* the width of display (LNEPIX; although not shown. 
LNEPIX is part of SPDSZ in FIG. 4 and indicates 
the number of dots on one line) is set at coding data 
separator 103. 

Address controller 109 sends the decoding 
addresses to memory controller 105. Then, the (tela to 
be decoded (La. corrpressed sub-picture data SPD) is 
read, via memory controller 105, from memory 108 to 
coding data separator 103 and to continuous code 
length detector 106. In this case, the read data is set al 
respective registers of separator 103 and detector 106 
in unit of byte (step ST1204). 

Continuous code length detector 106 courts the 
number of 0-bit of the data read from memory 108, and 
delects the coding header corresponding to either of 
rides 1 to 5 shown in FIG. 5 (step ST1205). Details ol 
detection of 1he coding header will be described later 
with reference to FIG. 20. 

Then, continuous code length detector 106 gener- 
ates separation information SEP.INFO. corresponding 
to either of rules 1 to 5 shown in FIG. 5 (step ST1206). 

More specifically, tor example, when the count of 0- 
brt of data read from memory 108 is zero, separation 
information SEP.INFO. indicating rule 1 is obtained; 
when the count of 0-bft of data read from memory 108 is 
two, separation information SEP INFO, indicating nde 2 
is obtained; when the count of 0-bit of data read from 
memory 108 is four, separation information SEP.INFO. 
inefcating rude 3 is obtained; when the count of 0-bit of 
data read from memory 108 is six, separation informa- 
tion SEP.INFO. indicating rule 4 is obtained; and when 
the count of (Kbit of data read from memory 108 is four- 
teen, separation information SEP.INFO. indicating rule 5 
Is obtained. The separation information SEP.INFO. this 
obtained is sent to coding data separator 103. 

In accordance with the content of separation infor- 
mation SEP.INFO. from continuous code length detector 
106, coring data separator 103 sets the number ol pix- 
els followed (PIXCNT; run information) at nn length set- 
ter 107. Further, coding data separator 103 sets at pixel 
color output stage 104 tie 2-bK pixel data (pixel color 
data picked ip from the sii>picture data packet) sii>se- 
quent to the number of pixels followed. At this time, ar- 
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rem count value NOWPIX of a pixel counter (not shown) 
in separator 103 is incremented by number PDCCNT of 
pixels -followed (step ST1207). 

FIG. 19 is a flow chart for explaining the latter 
processing (subsequent to node A in FIG. 18) of the s 
image decoding (run-length expansion) according to 
another embodiment of the present invention. 

In the former step ST1203, coding data separator 
103 is informed, by header separator 113. of the 
number LNEPIX of pixel data (the number of dots) of w 
one line corresponding to the display width of the sub- 
picture. Coding data separator 103 checks whether the 
value NOWPIX of the internal pixel counter of separator 

103 exceeds the value LNEPIX of the informed one-line 
pixel data (step ST1 208). 75 

In step ST1208, when the pixel count value 
NOWPIX is equal to or larger than the one-line pixel 
data value LNEPIX (NO at step ST1208), the internal 
register of separator 103, to which one byte data is set, 
is cleared, and the pixel count value NOWP IX becomes so 
zero (step ST1209). At this time, if the data has been 
byte-aligned, the tralfng 4-bit data is discarded or 
neglected. On the other hand, when the pixel count 
value NOWPIX is smaller than the one-line pixel data 
value LNEPIX (YES at step ST1 208), the internal regie- ss 
ter of separator 103 is not cleared, or the state of the 
internal register remains. 

Run length setter 1 07 responds to the number PDC- 
CNT (run information) of pixels followed set at the 
former step ST1 207, to dot clock DOTCLK determining 30 
the rate of transferring the pixel dots, and to horizontal- 
and vertical-sync signals H-SYNC and V-SYNC for syn- 
chronizing the sub-picture to the main picture display. In 
response to these data or signals, run length setter 107 
generates a display period signal (PERIOD SIGNAL) ss 
which is provided for enabling pixel color output stage 

104 to output for a required period of time the pixel data 
set in stage 104. The generated PERIOD SIGNAL is 
sent to pixel cola output stage 1 04 (step ST1210). 

Pixel color output stage 104 outputs, as display 40 
data of the decoded sub-picture, the separated data 
(e.g., pixel data indicating the pixel color) set at the 
former step ST1207 for a period during which the 
PERIOD SIGNAL is being sent from run length setter 
107 to output stage 104 (step ST1211). 45 

Thereafter, the sub-picture display data tfius 
obtained can be properly superirrposed to a main pic- 
ture by a specific circuit block (not shown), and an 
image of the superimposed main/sub-picture can be 
displayed on a TV monitor (not shown). so 

After the output processing of the pixel data at step 
ST1211 is comple te d, if the data to be decoded 
remains, the How returns to the former step ST1204 
(NO at step ST1212). Whether or not the data to be 
decoded remains can be determined by checking If cod- ss 
ing data separator 103 completes the data processing 
for the end address (SPEDADR) of the sub-picture dis- 
play data set by header separator 113. 



When no data to be decoded remains, or the data 
processing up to the end address is completed (YES at 
step ST1212), it is checked whether a display enable 
signal (DISPLAY ENABLE) from display activator 1 10 is 
active or not Note that activator 110 generates the 
active (e.g., high-level) display enable signal unless 
activator 110 receives a data end signal (DATA END 
SIGNAL) from address controller 109. 

Even when the decoding of current data has 
already been completed, H the display enable signal is 
stil active, rt is determined that the current stale is in the 
display period of the sub-picture (YES at step ST121 3). 
In this case, display activator 110 sends a color switch 
signal to run-length setter 107 and to pixel cdor output 
stage 104 (step ST1214). 

At the time when the color switch signal is sent, 
pixel color output stage 1 04 already receives insufficient 
pixel color data from insufficient pixel color setter 111. 
When output stage 1 04 receives the color switch signal 
from display activator 1 10, the pixel color data to be out- 
put is switched to the insufficient pixel color data 
obtained from insufficient pixel color setter 111 (step 
ST1215). Hi en, so long as the display enable signal is 
active (or in the loop of steps ST1213 to ST1 215) during 
the cfisplay period in which no sub-picture data to be 
decoded exists, the display area for the sub-picture is 
filled with the insufficient pixel color provided by insuffi- 
cient pixel color setter 1 1 1 _ 

On the other hand, when the display enable signal 
is inactive, it is determined that the display period for the 
decoded sub-picture elapses (NO at step ST1213). In 
this case, cfisplay activator 110 supplies microcomputer 
1 12 with an end status indicating that the decoding of 
the sii>picture(s) of one frame is completed (step 
ST1216). The decoding of the sub-picture(s) in one 
screen image (or one frame) is thus completed. 

FIG. 20 is a flow chart for exemplifying the content 
of the coding header detection step ST1205 shown in 
FIG. 18. The processing of the coding header detection 
can be executed by continuous code length detector 
106 shown in FIG. 17 (or FIG. 11). 

Rrst, continuous code length detector 106 is initial- 
ized so that its internal status counter (STSCNT; not 
shown) is set to zero (step ST1 301). Then, the contents 
of two-bit subsequent to the data which has been read 
from memory 108 to detector 106 in unit of byte are 
checked. When the contents are "00" (YES at step 
ST1302), counter STSCNT is incremented by 1 (step 
ST1303). tf the checked two-bit does not reach the end 
of one byte (NO at step ST1304), the contents of next 
two-bit are further checked. When the contents are still 
TXT (YES at step ST1302), counter STSCNT is further 
incremented by 1 (step ST1303). 

After repeating the processing of the loop of steps 
ST1S02 to ST1S04, if the checked twottt reaches the 
end of one byte (YES at step ST1304), it is determined 
that the coding header of FIG. 5 is constituted by more 
than 6 bits. In this case, continuous code length detec- 
tor 106 reads next one byte data from memory 108 
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(step ST1305), and status counter STSCNT is set to w 4" 
(step ST1307). At this time, the same one byte data is 
read by coding data separator 103. 

After status counter STSCNT is set to "4", or the 
contents of two-bit checked at the former step ST1302 s 
is not "00" (NO at step ST1302), the content of status 
counter STSCNT is established, and the established 
content is output as the content of the coding header 
shown in FIG. 5 (step ST1307). 

More spectficatly, when the established content of io 
6tatus counter STSCNT is "0", the coding header of rule 

I of FIG. 5 is detected. When STSCNT = T, the coding 
header of rule 2 is detected. When STSCNT = "2", the 
coding header of rule 3 is detected. When STSCNT « 
"3", the coding header of rule 4 is detected. When rs 
STSCNT = "4", the coding header of rule 5 (the same 
pixel data continues 14) to the end of fine) is detected. 

FIG. 21 is a flow chart for explaining the image 
decode processing according to the present invention, 
wherein the decoded image is scrolled. 20 

First the respective blocks in decoder 101 of FIG. 

II or FIG. 17 are initialized, and line counter LINCNT 
(not shown) is cleared to zero (step ST1401). Then, 
microcomputer 112 (FIG. 11) or header separator 113 
(FIG. 17) receives the header read end status sent at & 
step ST1201 of FIG. 18 (step ST1402). 

The content (zero at initial) of One counter LINCNT 
is sent to microcomputer 112 (FIG. 1 1) or header sepa- 
rator 113 (FIG. 17) (step ST1403). Microcomputer 112 
or header separator 1 13 checks whether the received so 
status is the end status (step ST1206 of FIG. 18) of one 
frame or one screen (step ST1404). 

If the received status is not the end status of one 
frame (NO at step ST1 405), the processing waits lor the 
end status. When the end status is received (YES at ss 
step ST 1405), fine counter UNCNT is incremented by 
one (step ST1 406). 

When the content of incremented counter LINCNT 
does not reach the end of One (NO at step ST1407), the 
decode processing of FIGS. 15 and 16, or the decode *o 
processing of FIGS. 18 and 19 is restarted (step 
ST1408), and the processing returns to step ST1403. 
To repeat the restart of decoding (steps ST 1403 to 
ST1408), the run-Jength-compressed sub-picture can 
be scrolled whOe it is decoded. 45 

Meanwhile, when the content of incremented coun- 
ter UNCNT reaches the end of line (YES al step 
ST1407), the decode processing associated with scroll- 
ing of the sub-picture is finished. 

FIG. 22 is a block tfiagmm for explaining the brief ao 
configuration of an optical disk record/payback appara- 
tus in which the encocing (of SPUH + PXD + DCSQT in 
Fia 3) and decoding (of SPUH + PXD + DCSQT) of the 
present invention are executed. 

Optical disk player 300 of FIG. 22 basically has the ss 
same configuration as a conventional optical disk play- 
back apparatus (such as a compact disk player or a 
laser disk player). However, optical cfisk player 300 has 
a special configuration that a digital signal, obtained 



before decoding the run-length-compressed image 
information (Le., an encoded digital signal), can be out- 
pul from inserted optical cfisk OD on which recorded is 
image information containing run-length-compressed 
sub-picture data according to the present invention. 
Since the encoded digital signal is compressed, the 
transmission bandwidth necessary to the encoded dg- 
rta! signal can be narrower than that necessary to non- 
compressed data. 

The compressed digital signal from optical disk 
player 300 is on-aired or is output to a communication 
cable, via modulator/transmitter 210. 

The on-aired compressed digital signal or the 
cable-output compressed digital signal is received by 
receiver/demodulator 400 of a user or subscriber. 
Receiver 400 is provided with decoder 101 having a 
configuration as shown, for example, in FK3L 1 1 or FIG. 
17. Decoder 101 of receiver 400 decodes the com- 
pressed digital signal having been received and demod- 
ulated, so as to output image information containing the 
original sib-picture data before encoded. 

In the configuration ol FIG. 22, when the transmis- 
sion system of the transmitter/receiver has an average 
bit rate of more than about SM-bitfe, htgh-qualrty multi- 
media videc/audio information can be transmitted. 

FIG. 23 is a block diagram for explaining a case 
wherein image information encoded on the basis of the 
present invention is exchanged between two arbitrary 
computers through a communication network (e.g., an 
internet). 

User #1 having user's source #1 , which is managed 
by a host computer (not shown), has personal computer 
5001. Various input/output devices 5011 and various 
external memory units 5021 are connected to computer 
5001 . Modem card 5031 incorpor atin g the encoder and 
the decoder of the present invention and having a func- 
tion required for communication is inserted in an inter- 
nal slot (not shown) of personal computer 5001 . 

Similarly, user #N having user's source #N has per- 
sonal computer SOON. Various input/output devices 
501 N and various external memory units 502N are con- 
nected to computer 500N. Modem card 503N incorpo- 
rating the encoder and the decoder of the present 
invention and having a function required for communica- 
tion is inserted in an internal slot (not shown) of per- 
sonal computer SOON. 

Assume that user #1 operates computer 5001 to 
communicate with computer SOON of another user #N 
through line 600 such as an internet In this case, since 
both users #1 and #N have modem cards 5031 and 
503N incorporating the encoders and the decoders, 
compressed image data can be efficiently exchanged 
within a short period of time. 

FIG. 24 shows a brief configuration of a 
recorttylayback apparatus for recording on optical disk 
OD the image inf or m a t io n (SPUH + PXD + DCSQT in 
FIG. 3) encoded acoorcfing to the present invention, and 
for playing back the recorded information (SPUH + PXD 
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+ DCSQT) to decode it according to the present inven- 
tion. 

Encoder 200 of FIG. 24 is so constructed that it per- 
forms the encode processing (corresponding to the 
processing of FIGS. 13 and 14) similar to the encode 5 
processing of encoder 200 of FIG. 10, provided that 
encoder 200 of FIG. 24 executes the encode processing 
based on a software or hardware (containing a firmware 
or wired-togic circuits). 

The record signal containing sub-picture data to 
encoded by encoder 200, etc. is subjected to, for exam- 
ple, a (2.7) RLL modulation at modulator/laser driver 
702. The modulated record signal is sent from laser 
driver 702 to a high-power laser diode mounted in opti- 
cal head 704. A particular pattern correspond ng to the is 
record signal is written in a magneto-optical disk or 
phase-change optical disk OD by means of the record- 
ing laser from optical head 704. 

Thereafter, the information written in disk OD is 
read by a laser pickup of optical head 706. TTie read so 
information is then demodiiatBd at demodulator/error 
correction circuit 708 in which an error correction is per- 
formed, if necessary. The demodulated and error-cor- 
rected signal is subjected to various data processing at 
data processor 710 tor aucftoArideo information, so that 2s 
information, equivalent to the original information before 
recording, is played back. 

Data processor 710 includes a decode processing 
portion corresponding to, for example, decoder 101 of 
FIG. 11. This decode processing portion can perform ao 
the decocfing (Le. ( expanding of compressed sub-pic- 
ture data) of FIGS. 15 and 16. 

FIG. 25 shcnvs an example of an IC device in which 
the encoder of the present invention is integrated with 
its peripheral circuits. ss 

FIG. 26 shows an example of an IC device in which 
the decoder of the present invention is integrated with 
its peripheral circuits. 

FIG. 27 6hows an example of an IC device in which 
the encoder and decoder of the present invention are to 
integrated with their peripheral circuits. 

Thus, the encoder andfor the decoder accortfing to 
the present invention can be reduced to practice in a 
form of a semiconductor integrated circuit (IC)- The 
present invention can be embodied by installing such an *s 
IC in various apparatuses. 

Generally, the data line on which the bit train of 
compressed data (PXD) as shown in FIG. 9 is arranged 
includes 1he image information of one horizontal scan- 
ning line of a TV display screen. However, the data tine so 
may include the image information of two or more hori- 
zontal scanning lines of the TV display screen, or may 
include the Image information of afl horizontal scanning 
lines of one TV display screen (or one frame of the TV 
display screen). ss 

Incidentally, the target of data encoding based on 
the compression rules of the present frivention is not 
limited to the sub-picture data (color information with 
respect to 3 or 4 colors) explained in the present speci- 



fication. When the portion of the pixel data is constituted 
by more bit numbers, various information can be packed 
in this portion. For instance, if 8-bit per one pixel dot is 
assigned to the pixel data, 256-cotor pictures can be 
transmitted only by tie sub-picture (in addition to a main 
picture). 

Sifc-picture data as shown in FIG. 2 or 3 has a pJu- 
rafity of channels, as shown in FIG. 52. The sub-picture 
data block is constituted by a plurality of sub-picture 
data packets of a channel arbitrarily selected from the 
plurafity of channels. The sub-picture is information of, 
e.g., a character or figure. The sub-picture Is played 
back simultaneously with video and audio data and is 
superposed on the playback frame of the video data. 

FIG. 29 is a view showing the structure of a sub-pic- 
ture data packet As shown in FIG. 29, the sub-picture 
data packet has packet header 3, sub-picture header 
31, sub-picture content data 32, and display control 
sequence table 33. 

The time at which a playback system starts control- 
ling a sub-picture data block is recorded as a time 
stamp (to be referred to as a PTS; Presentation Time 
Stamp) in packet header 3. Note that the PTS is 
recorded in only header 3 of the start sub-picture data 
packet in each of cub-picture data blocks (Y, W) . 

FIG. 30 exemplifies a series arrangement (n, n+1) 
of sub-picture units (cf. 30 in FIG. 3) each of which is 
constituted by one or more sub-picture packets; time 
stamp PTS described in the packet header of one unit 
(n+ 1 ) ; and conditions (clear of the display of the preced- 
ing sub-picture, and designation of the display control 
sequence of the sub-picture to be displayed from now) 
of display control of the unit (n+1) corresporrcfing to the 
PTS. 

The size (SPDSZ of 2 bytes) of a sub-picture data 
packet, and the recording start position (SPDCSQTA of 
2 bytes) of display control sequence table 33 in the 
packet, are recorded in sub-picture header 31. 

At least one display control sequence information 
(DCSQT; Display Control Sequence Table) having, as 
one group, a sub-picture display control firm stamp (to 
be referred to as SPDCTS; Sub-Picture Display Control 
Time Stamp) representing the display start/end time of 
the sub-picture data, the recording position (SPNDC- 
SQA; Sub-Picture Next Display Control Sequence 
Address) of sub-picture content data (PXD) 32 to be dis- 
played, and the display control command (COMMAND) 
of the sub-picture content data is recorded In display 
control sequence table 33. 

Time stamp PTS in packet header 3 is defined by 
the time relative to a reference time (SCR; System 
Clock Reference) for playback of an overall fie, e.g., by 
the playback start time at the beginning of a fie (cf. FIG. 
2). Meanwhile, each SPDCTS in display control 
sequence table 33 Is defined as the time relative to the 
PTS 

Time stamp PTS processing of a sub-picture data 
packet In the playback system will be described below. 
Assume that the PTS processing is executed by a sub- 
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picture processor (such as MPU 1 12 and its peripheral 
circuitry) in the playback system. 

FIG. 52 explains how the state of storing sub-pic- 
ture data blocks is changed with sub-picture channels 
each having time stamp PTS, when the sub-picture data s 
is decoded. 

(1) The sub-picture processor (cf. FIGS. 11, 17, 
etc.) decodes each sub-picture data packet of a 
selected channel from externally input sub-picture io 
data packets (delivered from an optical disk, a 
broadcasting station, etc), so as to check the pres- 
ence/absence of a PTS in the decoded packet 

If the PTS is present as indicated by channel 
*4f in FIG. 52, this PTS is extracted from packet rs 
header 3. The PTS is added to the beginning of the 
sub-picture data as shown in FIG. 28. The resultant 
sub-picture data having a header with PTS is buff- 
ered (or stored) in a sii>picture buffer (e.g.. buffer 
121 in FIG. 11). 20 

Note that the graph of FIG. 52 illustrates how 
the buffering amount of sub-picture buffer 121 is 
accumulated as the sub-picture data packets of 
channel *4f with PTS are buffered. 

(2) After the 6ystem is reset, the sub-picture proc- 2s 
essor loads the PTS during a vertical blanking 
period (during which a tSsplay frame/lield is 
switched to the next display frame/field) immedi- 
ately succeeding reception of the first packet 
including this PTS. The sub-picture processor com- so 
pares this PTS with an STC. The STC represents 
the value of a stand a rd time counter (e.g., part of 
timer 120 in FIGL 1 1) in the sub-picture processor, 
which measures the lapse of time from the refer- 
ence time SCR for playback of the overall file. as 

(3) As a result of comparison between the PTS and 
STC, if the STC is larger than the PTS, this sub-pic- 
ture data is immediately subjected to display 
processing. However, if the STC is smaller than the 
PTS, no processing is performed. This comparison 40 
is performed again in the next vertical blanking 
period. 

(4) In the display processing for the sub-picture 
data, the first sub-picture display control time stamp 
SPDCTS recorded in display control sequence 4s 
table 33 in this sub-picture data packet is compared 
with a substandard time counter (sub-STC) in the 
sub-picture processor during the same vertical 
blanking period. Here, the sub-STC is constituted 

by a siJ>etandard time courier (e.g„ other part of so 
timer 120 in FIGL 11) in the sub-picture processor, 
which measures the lapse of time from the playback 
start time of the sub-picture data block Thus, all the 
bits of the sub-STC are cleared to "O^s every time 
the display Is switched to the next sub-picture data ss 
block, and the sub-STC is incremented (i.e., time- 
counted) again. 

(5) As a result of comparison between the sub-STC 
and sub-picture display control time stamp SPD- 



CTS, if the sub-STC is larger than the SPDCTS, 
control data (DCSQT; e.g., DCSQ 0 in FIG. 29) of 
the first display control sequence in display control 
sequence table 33 is immediately executed, and 
display processing for the si&-picture is started. 

(6) Once the display processing is started, the PTS 
added to the first packet in the next sub-picture data 
block in the currently displayed sub-picture data 
block is loaded every vertical blanking period. This 
PTS is compared with the count of standard time 
counter STC. 

As a result of this comparison, if the STC is 
larger than the PTS, the channel pointer of FIG. 29 
is set to indicate the PTS address value of the mod 
sub-picture data block, and the sub-picture data 
block to be processed is switched to the next one. 
For example, in FIGL 28, sub-picture data Mock Y is 
switched to next sub-picture data block W. At this 
time, since the data of sib-pictura data block Y is 
no longer used, an empty area corresponding to the 
size of sLfc-picture data block Y is formed in the 
sub-picture buffer (e.g., memory 108 in FIG. 11), so 
that new sub-picture data packets can be trans- 
ferred to this empty area. 

The buffered state (cf. FIG. 52) of the sub-pic- 
ture data packet can be primarily defined in 
advance at the time of encoding the sub-picture 
data, in accordance with the size of the sub-picture 
data block (e.g.. block Win FIG. 28) and the switch- 
ing time thereof (eg., the time at which block Y is 
switched to block W). Therefore, in serial transfer of 
video picture, audio, and sub-picture packets, it is 
possible to generate a bit stream which causes nei- 
ther an overflow nor underflow in the buffers of the 
respective decoders (e.g., memory 108 in FIGL 11, 
etc, for the sub-picture decoder). 

As a result of the comparison between the PTS 
and STC, If the STC is not larger than the PTS, sub- 
picture data block switching is not performed, and 
the cfisplay control sequence table pointer (DCSQT 
pointer in FIG. 29) is set in the next DCSQT 
address value. The sub-picture cfisplay control time 
stamp SPDCTS of the next DCSQT in the current 
sub-picture data packet is compared with the sub- 
STC. Whether the next DCSQT is executed is 
determined based on the comparison result This 
operation wfll be described in detail later on. 

Note that the DCSQT processing in (5) for the 
last DCSQT in the sub-picture data packel remains 
basically the same, because the display control 
sequence table DCSQT designates itself as the 
next DCSQT 

(7) In a normal playback (or reproduction) opera- 
tion, processing of (4), (5), and (6) is repeated. 

In processing operation (6). the channel pointer 
(cf. FIG. 29) representing a PTS in loading this PTS 
in the next sub-picture data block is obtained using 
the packet size (SPDSZ) of the current siiy-picture 
data block. 
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Similarly, the value of the DCSQT pointer indi- 
cating sub-picture display control time stamp SPD- 
CTS of next DCSQT in display control sequence 
table 33 can be obtained horn the size information 
(address SPNDCSQTA of the next sub-picture dis- 
play control sequence) described in table 33. 

Sub-picture header 31 , sub-picture content data 32, 
and display control sequence table 33 will be described 
in detail below. 

FIG. 31 is a view showing the structure of the sub- 
picture unit header (SPUH) 31. Sub-picture unit header 
SPUH 31 includes the size (SPDSZ) of a sub-picture 
data packet, and the recording start position (SPDC- 
SQTA; relative address pointer of DCSQ) of display con- 
trol sequence table 33 in the packet 

Incidentally, as shown in FIG. 32, the contents of 
sub-picture display control sequence table SPDCSQT 
indicated by adcfress SPDCSQTA are constituted by a 
plurality of display control sequences DCSQ1 to 
DCSQn. 

Further, as shown in FIG. 33, each of display con- 
trol sequences DCSQ1 to DCSQn includes sub-picture 
display control time stamp SPDCTS indicating the sub- 
picture display control start time, address SPNDCSQA 
indicating the position of the next display control 
sequence, and one or more sub-picture display control 
commands SPDCMD. 

Sub-picture content data 32 is an agfpegate of data 
areas (PXD areas) which have a one-to-one corre- 
spondence with the individual sub-picture data packets. 

More spedlicalty, untd the sub-ptctire data block is 
switched, sub-pictm pixel data PXD at an arbitrary 
address in the same data area is readout For this rea- 
son, an arbitrary display such as a scroll display, which 
is not fixed to one display image, can be performed. 
This arbitrary address is set by a command (e.g., 
SET.DSPXA in the command table of FIG. 34) lor set- 
ting the display start address of the sub-picture content 
data (pixel data PXD). 

FIG. 43 shows a brt-corrfiguration of command 
SET_DSPXA, which is one of the commands exempli- 
fied in FIG. 34, for setting the display start address of 
the pixel data of sub-picture. In the following, meaning 
of the configuration of this command will be described. 

When the line data sizes of sifc-picture lines con- 
tained in sub-picture content data 32 are different from 
one another, the start address of the next line can be 
discriminated only after the immediately preceding line 
data is decoded. Then, when Image data are arranged 
in the line number order as in a conventional case, it is 
difficult to read pixel data (PXD) from a buffer (memory 
1 06) while skipping one line in the interlace mode. 

In view of this problem, as shown in FIG. 58, sub- 
picture content data 32 are separately recorded in top 
and bottom field areas 61 and 62 in units of data areas 
corresponding to the individual sub-picture data pack- 
ets. In the Interlace mode, two, top and bottom field 
address fields 63 and 64 are given to command 



SET_DSPXA 60 as to allow setting of two start 
addresses, i.e., the top and bottom field start 
addresses. 

Note that, in the nonmterlace mode, sub-picture 
s content data for one field is recorded, and the same 
address is recorded in both address fields 63 and 64 of 
the top and bottom fields. 

FTG. 59 is a view showing the format of display con- 
trol sequence table 33. As described above, a plurality 
10 of display control commands (COMMANDS, COM- 
MAND 4, etc.) and respective parameters set by these 
commands are arranged after the time stamp (SPD- 
CTS) and the sub-picture content data recording posi- 
tion (SPNDCSQA) In one display control sequence 
75 information (DCSQT) of display control sequence table 
33. Finally, an end command (End_Code) representing 
the end of display control is added. 

The processing sequence of this display control 
sequence table 33 wil be described below. 

20 

(1) First, the time stamp (SPDCTS) recorded in the 
first DCSQT (DCSQTO in FIG. 23) in display control 
sequence table 33 is compared with the sub-STC 
(e.g., a function of timer 120 in FIG. 1 1) of the sub- 

25 picture processor. 

(2) As a result of this comparison, if the sub-STC is 
larger than the lime stamp SPDCTS, all the display 
control commands in display control sequence 
table 33 are executed until display control end com- 
ae? mand CMD_END (cf. FIG. 34) appears. (3) After 

the start of the display control, the sub-picture dis- 
play control time stamp SPDCTS recorded in the 
next display control sequence table DCSQT is com- 
pared with the sub-STC every predetermined 

as period of time (eg., every vertical blanking period) 
to determine whether updating to the next DCSQT 
is to be performed. 

Time stamp SPDCTS in display control 
sequence table 33 is recorded as a relative time 

40 with respect to updating of the PTS, (or updating of 
the sub-picture data block). For this reason, even if 
the PTS of the sub-picture data packet changes, 
the SPDCTS need not be rewritten. Even If the 
same sub-picture content delta 32 isto becfisplayed 

45 at effferent times, the same display control 
sequence table DCSQT can be used, so that relo- 
catabifity of the DCSQT can be guaranteed. 

The cfisplay control commands tor sub-picture will 
so be described in detail below. 

FIG. 34 shows a set of sub-picture display control 
co mma nds SPDCCMD. Major commands thereof are 
as follows: 

55 (1 ) PixbI Data Display Start Timing Setting (STAJDSP) 

FIG. 37 shows the format of command STA_PSP. 
This Is a command far executing display start control for 
sub-picture content data 32. More specifically, when the 
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DCSQT is switched from a given DCSQT to another 
DCSQT inclucfing thi8 command STA_DSP, the display 
of the sub-picture content data 32 is started from the 
time represented by the time stamp (SPDCTS) of the 
DCSQT including this command. s 

When the sub-picture processor (eg., MPU 11 2 of 
FIG. 1 1) decodes this command, it immediately renders 
active the enable bit of an internal display control sys- 
tem of the subpicture processor (because when this 
command Is accessed, the time represented by the w 
SPDCTS of the DCSQT to which this command belongs 
has passed). 

(2) Pixel Data Display End Timing Setting (STPJDSP) 

15 

FIG. 38 shows the format of command STP_DSP. 
This is a command for executing display end control for 
the sub-picture content data 32. When the sub-picture 
processor decodes this command, fit immediately 
renders inactive the enable bit of the internal display 20 
control system of the sub-picture processor (because 
when the command is accessed, the lime represented 
by the SPDCTS of the DCSQT to which this command 
belongs has passed). 

25 

(3) Pixel Data Color Code Setting (SETCOLOR) 

FIG. 39 shows the format of command 
SET_COLOR. This is a command for setting the color 
code of pixel data. By this command, odor information 30 
can be sat into a pattern pixel (e.g., a character or pat- 
tern), an emphasized pixel (e.g., edging for a pattern 
pixel), and a background pixel as a pixel except for the 
pattern pixel and the emphasized pixel in the range in 
which a sUb-picture is displayed 35 

As shown in FIGL 40, the sub-picture processor 
incorporates color register 1210 capable of setting a 
color code by command SET_COLOR. Once the color 
code is set by this command, color register 1210 holds 
the color code until another color code is set again using 40 
the seme command. Thus, selected (SELO) from color 
resister 1 2 1 0 is the color data based on a type of pixels 
(ag.. the type specified by the 2-bit pixel data ol FIG. 5) 
indicated by sub-picture content data 32. 

The sub-picture processor also incorporates 4S 
change color data register 1220 which is rat by a com- 
mand (CHG_COLCON) for setting changes in color and 
contrast of the sii>-picture pixel data. When the data 
output selected (SELO) from register 1220 is active, this 
data output is then selected (SEL1) and output as a bo 
selected color data, since the selected data output from 
register 1220 has a higher priority than that of the 
selected data output from register 1210. 

(4) Pixel Data Main Pichre Contrast Setting ss 
(SET — CONTR) 

FIG. 41 shows the format of command 
SET — CONTR. This is a command for setting contrast 



data, in the same manner as in the case of command 
SET.COLOR, in place of color code data for the four 
types of pixels described with reference to FIG. 40. 

(5) Pixel Data Display Area Setting (SET.DAREA) 

FIGL 42 shows the format of command 
SET.DAREA. This is a command for setting the display 
position of sub-picture pixel data 32. 

(6) Sub-Picture Content Data Use Range Setting 
(SETJ5SPXA) 

FIGL 43 shows the format of command 
SET_DSPXA_ This is a command for setting the start 
address of sub-picture pixel data. 

(7) Pixel Data Color Change and Contrast Change Set- 
ting (CHG.COLCON) 

FIG. 44 shows the format of command 
CHG_COLCON. This is a command for changing, dur- 
ing the display, the color of sub-picture pixel data 32 as 
well as for changing the contrast of sub-picture pixel 
data 32 with respect to the main picture. 

This command CHG_COLCON nckides the pixel 
control data (PCD) and its size (extended field size). 

incidentally, in addition to the above-mentioned 
commands, the command table shown In FIG. 34 also 
includes command FS1AJDSP (cf. FIG. 36} for forcftxy 
setting the display start timing of the sub-picture pixel 
data, and command CMD_END (cf. FIG- 45) for termi- 
nating the display control of the sub-picture. 

FIGS. 35, 46, and 47 explain the format or configu- 
ration of pixel control data PCD. Pixel control data PXD 
has line control information LCINF, pixel control informa- 
tion PCINF, and a termination code (or end code) of 
pixel control data. 

As shown in FIG. 46, line control information LCINF 
is constituted by a change start line number, the number 
of changes (a change point count), and a change termi- 
nation line number (or continuous 6ne count). 

More specifically, line control information LCINF 
incicates the start fine of a cfi splay frame for control of 
an outline correction color, a sub-picture color, and the 
contrast of the sub-picture with respect to the main pic- 
ture; the number of changes of the outfine correction 
color, the sub-picture color, and the contrast on these 
lines; and an end of the line at which the common 
change lasts. 

With respect to the line being indicated by line con- 
trol information LCINF, pixel control information PCINF 
represents the pixel position at which thB outline correc- 
tion color, the sub-picture color, and the contrast are to 
be changed; and the contents of changes of the outline 
correction color, the sub-picture color, and the contrast 
upon the change. 

The number of pixel control data each constituting 
fine control information LCINF and pixel control informa- 
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tion PCINF is set for sub-picture display frames, as 
needed. 

For example, pixel control data PCD set for a sub- 
picture display frame image of FIG. 48 can be 
expressed as shown fin FIG. 19. 

More specifically, in the above example, a fine on 
which a change is started is line 4. THb change start line 
number is 4. The positions at which pixels change are 
positions A, B, and C The pixel change point court is 3. 
Since the state of common pixel change continues ip to 
line 11, the continuous line count is 7. 

Line 12 has a pixel change state different from that 
on the previous lines. No pixel change occurs on One 13. 
Another f ne control information is sat such that the 
change start fine number is 12, the change point count 
is 2, and the continuous line count is 1 . Line 14 has fair 
positions at which pixBls change. No change in pixel 
occurs in next line 15. For this reason, another line con- 
trol information LCINF is set such that the change start 
line number is 14, the change point count is 4, and the 
continuous line count is 1 . Finally, an end code repre- 
senting the end is set. 

A display control sequence using line control infor- 
mation LCINF and pixel control information PCINF will 
be descra>ed below. 

(1) Display control of sub-picture is performed by 
repeated executions of control commands 
(COMMAND1, etc.) contained in display control 
sequence table 33 (cf. DCSQT1 to DCSQTN of 
FIG. 29). These command executions are repeated 
in each of the display fields. The contents of the 
control commands are shown in the table of sub- 
picture display control command SPDCCMD in 
FIG. 34. 

Which display control sequence (any of 
DCSQT1 to DCSQTN) is selected to execute its 
command (of various commands in FIG. 34) can be 
determined by the DCSQT pointer shown in FIG. 
29. 

(2) The parameters set by respective control com- 
mands (STAJDSP, STP_DSP, SET_COLOR, 
SETJSONTR SETLDAREA. SETJDSPXA. 
CHG_COLCON) shown in FIG. 34 are Kept held in 
the corresponding registers in the sub-picture proc- 
essor (ag., MPU 112 in FIGL 11), unless these 
parameters are rewritten by the same command. 
Except for specific parameters (LCINF, PCINF), 
respective parameters held in the above registers 
are cleared when the sub-picture data block is 
switched (ag., switched from block Y to block W in 
FIG. 28). 

More specifically, the parameters (LCINF, 
PCINF) of pixel control data PCD in FIG. 35 are 
held in the internal registers of MPU 112, unless 
command CHG_COLCON of FIG. 34 is executed 
again. 

(3) In a highlight mode, display control is performed 
by the LCINF and PCINF parameters set by system 



MPU 112. The LCINF and PCINF of the sub-picture 
content data 32 are neglected in this case. These 
set parameters (LCINF and PCINF) are kept held 
and the display using the held LCINF and PCINF 
s continues until they are set again by system MPU 
112 in the highlight mode, or the LCINF and PCINF 
in the sub-picture data are set again in the normal 
mode. 

(4) The display area is set by lines and dots whose 
70 numbers are designated at the start and and in both 
the horizontal and vertical directions. To display 
only one Gne, the di^lay start line number must be 
equal to the display end line number. If no display is 
to be performed, the display must be stopped using 
75 a display end command. 

FIG. 53 is a flowchart for explaining a manner of 
generating sub-picture unit 30 as shown in FIGL 3. 
When captions and/or images corresponding to the 

so speech or words of the video (main picture) are used for 
the six>picture, the captions and/br images are con- 
verted into bit-mapped data (step ST1 0). When the bit- 
mapped data is prepared, it is necessary to determine 
where the part of the captions anoVor images is to be 

25 displayed within the display screen of the video (main 
picture). To determine the location of display of the cap- 
tions and/or images, the parameter of display control 
command SET_DAREA (cf. FIG. 34) is determined 
(step ST12). 

so After the display location (spatial parameter) of the 
sub-picture (bit-mapped captions andfor images) is 
determined, pixel data PXD constituting the bit-mapped 
sub-picture is encoded. (Thus, all part of the video are 
not encoded; details of the encoding of PXD are already 

35 explained with reference to FIGS. 5 to 1 4.) At this time, 
the color of the captions (sub-picture), the background 
color of the caption area, the ratio of mixture of the cap- 
tion color and background color with respect to the 
video main picture are determined. For this purpose, the 

40 parameters of display control commands SETCOLOR 
and SETjCONTR (cf. FIG. 34) are determined (step 
ST14). 

Thereafter, the timing of display of the prepared bit- 
mapped data with respect to the speech or words of the 

45 video is determined. This timing can be determined by 
sub-picture presentation time stamp PTS. At this tine, 
the maximum time length of time stamp PTS as well as 
respective parameters (time parameter) of display con- 
trol commands STA_DSP, STPJDSP, and 

so CHG_COLCON (cf. FIG. 34) are determined (step 
ST16). 

Here, sub-picture time stamp PTS is f inaly deter- 
mined according to the consumption model of a target 
decoder buffer of the MPEG2 system layer. In this 
55 embodiment the time of start of the caption display Is 
used for determining the maximum time length of time 
stamp PTS. 

Display control commands STA_DSP and 
STP_DSP are recorded as relative time information with 
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respect to &ub-picture time stamp PTS Thus, unless 
the content of PTS is determined, cfisplay control com- 
mands STA_DSP and STP_DSP carmol be deler- 
mined. For this reason, in the present embodiment, an 
absolute time system is adapted to determine the abso- s 
lute time of PTS. When the absolute time of PTS is 
determined, the relative time with respect to this abso- 
lute time can be determined. 

Further, when the display color and/or the display 
area of the prepared caption are to be changed in spa- 
tial domain or in time domain, the parameter of com- 
mand CHG_COLCON is determined according to the 
change of the prepared caption. 

After the display position (spatial parameter) and 
the display timing (time parameter) are (temporarily) 
determined, the content (DCSQ) of sub-picture display 
control sequence table DCSQT is prepared {step 
ST1S). More specifically, the value of display control 
start time SPDCTS (cf. FIG. 33) of display control 
sequence table DCSQ is determined in accordance with 
the time of issuing display control command STA_DSP 
(display start timing) and with the time of issuing display 
control command STPJ3SP (display end timing). 

The size of sub-picture data unit 30 (cf. FIG. 3) can 
be determined according to a combination of prepared 
pixel data PXD 32 and prepared display control 
sequence table DCSQT 33. Then, parameter SPDSZ 
(sub-picture size; cf. FIG. 31) of sub^xcture unit header 
SPUH 31 and SPDCSQTA (start address of display 
control sequence table; cf. FIG. 31) are determined 
according to the above determined size, so that sub-pic- 
ture urrit header SPUH 31 is prepared. Thereafter, the 
sub-picture lot It for one caption is prepared by confin- 
ing SPUH 31 . PXD 32, and DCSQT 33 (step ST20). 

When the size of prepared sub-picture unit 30 
exceeds a predetermined value (2043 bytes or 2k 
bytes) (yes at step ST22), prepared sub-picture unit 30 
is divided into one or more packets in units of 2k bytes 
(step ST24). In this case, presentation time stamp PTS 
is recorded only at the leading (or first) packet of pre- 
pared sub-picture urrit 30 (step ST26). 

When the size of prepared sub-picture unit 30 does 
not exceed the predetermined value (2k bytes) (no at 
step ST22), only one packet is prepared (step ST23), 
and time stamp PTS is recorded at the top of the pre- 
pared one packet (step ST26). 

One or more packets thus prepared are packed and 
contained with a video or other packs, so that a data 
stream is generated (step ST28). 

In this rase, the order of arrangement of respective 
packs is determined, based on the consumption model 
of a target decoder buffer of the MPEG2 system layer, in 
accordance with the sequence record coda SRC and 
sub-picture time stanp PTS. Time stamp PTS is finally 
determined here, and respective parameters (SPDCTS. 
etc) of FIG. 33 are also finally determined 

FIG. 54 is a flowchart to explaining an example of 
parallel-processing of the pack-disassembling of sub- 



picture data stream and decoding thereof, which data 
stream is encoded according to the process of FIG. 53. 

First, the decoding system reads the ID of the trans- 
ferred data stream, so that only selected sub-picture 
packs (separated from the data stream) are sent to the 
sub-picture decoder (e.g., sub-picture decoder 101 in 
FIG. 11 or FIG. 17) (step ST40). 

When the first pack is transferred, index parameter 
T is set to "1" (step ST42). and the disassembling for 
the first sub-picture pack is executed (step ST44; see 
FIG. 55)- 

The disassembled pack (which contains com- 
pressed sub-picture data PXD as shewn at the lower 
portion of FIG. 9) is temporally stored in the sii>-pictuTe 
buffer (e.g., memory 108 in FIG. 11 or FIG. 17) (step 
ST46), and index parameter V is incremented by "1" 
(step ST50). 

When the incremented i-th pack exists, or the pack 
disassembled at preceding step ST44 is not the final 
pack (no at step ST52), the disassembling for the incre- 
mented i-th sub-picture pack is executed (step ST44). 

The disassembled i-th sub-picture pack (eg., the 
second pack) is temporarily stored in the sub-picture 
buffer (or memory 108) as in the case cf the first disas- 
sembled pack (step ST4€), and index parameter T is 
further incremented byT (step ST50). 

As mentioned above, a plurality of sub-picture 
packs are continuously tisassembled (step ST44) while 
index parameter T is sequentially incremented, and 
disassembled packs are stored in the sub-picture buffer 
(or memory 108) (step ST46). 

When continuously incremented i-th pack is not 
found, or the pack disassembled at preceding step 
ST44 is the final pack (yes at step ST52), the sub-pic- 
ture pack disassembling process of the stream to be 
decoded is ended. 

During the continuous executions (steps ST44 to 
ST52) of the sub-picture pack disassembling process, 
decoding of sub-picture packs temporarily stored in the 
sub-picture buffer (memory 10S) is performed in parallel 
and independently with the sub-picture pack disassem- 
bling process. 

More specifically, when index parameter T is set to 
"1" (step ST60), the first sub-picture pack is read from 
the sub-picture buffer (memory 108) (step ST62). If the 
first sub-picture pack is not yet stored in the sub-picture 
buffer (memory 108) (no at step ST63; the process of 
step ST46 is not yet corrpleted at this lime), the decod- 
ing process executes an empty loop of the pack reading 
operations (steps ST62 to STBS) until the target pack 
data to be read is stored in memory 108. 

When thefiistsiJb-picture pack is stored in the sub- 
picture buffer (memory 108) (yes at step STBS), the first 
sub-picture pack is read therefrom, and subjected to the 
decoding (step ST64; see FIGS. 53 to 57). 

The result of the deaxfng (which contains uncom- 
pressed sub-picture data PXD as shown at the upper 
portion of FIG. 9) is sent to a display system (not shown) 
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from sub-picture decoder 101 in FIG. 11 or FIG. 17 dur- 
ing execution of the decoding. 

If the cfisplay control end command (GMDJEND in 
FIG. 34} is not yet executed in the above decoding proc- 
ess (no at step ST66), Index parameter T is incre- 
mented by *r (step ST67). 

When the Incremented f-th pack fin this case, the 
second sub-picture pack) is stored in memory 108 (yes 
at step STB3), the pack is read from memory 108, and 
subjected to the decoding (step ST64). The decoded j- 
th sub-picture pack (here the second pack) is sent to the 
display system as in the case of the first pack, and index 
parameter y is further incremented by "1" (step ST67). 

As mentioned above, one or more sub-picture 
packs stored in memory 108 are continuously decoded 
(step ST64) while index parameter T is incremented 
(step ST67). and an image display of the sub-picture 
corresponding to the decoded sub-picture data (PXD) is 
executed. 

During the above decoding process, if the display 
control end command (CMD_END in FIG. 34} is exe- 
cuted (yes at step ST66), the decoding process for the 
sub-picture stored in memory 108 is ended. 

The above decoding process (steps ST62 to ST64) 
is repeated unless end command CMD_END is exe- 
cuted (no at slap ST66). In the present embodiment, the 
decoding process is terminated by the execution of end 
command GMDJEND (yes at step ST66). 

FIG. 55 is a flowchart for explaining an example of 
the pack disassemble processing of FIG. 54. 

Sub-picture decoder 101 skips the pack header (cf. 
FIG. 3) in the pack sent from an outside to obtain the 
packet (step ST442). When the obtained packet does 
not contain presentation time stamp PTS (no at step 
ST444), the packet header (PH) is deleted from the 
obtained packet and only the sub-picture unit data 
(PXD) is stored in the buffer (e g , 121) of the sub-pic- 
ture decoder (step ST446). 

When the obtained packet contains presentation 
time stamp PTS (yes at step ST444), only the PTS is 
exh acted from the packet header (PH), and the 
extracted PTS is connected to the sub-picture unit data 
(30). The sub-picture unft data with PTS is then stored 
in buffer 121 of sub-picture decoder 101 (stepST446). 

FIG. 56 is a flowchart for explaining an example cf 
the sub-picture decode processing of FIG. 54. 

Sub-picture decoder 101 compares time SCR of 
system timer 120 with tine stamp PTS stored in buffer 
121 (6tep STB40). When they match (yes at step 
ST642), decoding of the corresponding sub-picture unit 
(30) starts. In the decoding process, how the com- 
pressed data PXD as shown at the lower portion of FIG. 
9 is restored to the uncompressed data PXD as shown 
at the ipper portion of FIG. 9 is already explained in 
connection with FIGS. 15, 16, etc 

In the decocf ng process, the respective commands 
of c£ splay control sequence DCSQ are executed. More 
specifically, the display position and cDsplsy area of the 
sub-picture are set by command SET_DAREA; the dis- 



play color of the sub-pi aire is set by command 
SETjCOUOR; the contrast of the sub-picture with 
respect to the video main picture is set by command 
SETCOMTR (step ST646). 

s After display start timing command STA.DSP is 
executed, and before cfsptay end timing command 
STP_DSP is executed in another display control 
sequence DCSQ, the decoding of run-length com- 
pressed pixel data PXD (32) is performed (step ST646) 

io while the display cortrol based on switch command 
CHG.COLCON is performed. 

incidentally, steps ST644 and ST646 are skipped 
when time SCR of system timer 120 does not match 
time stamp PTS stored In buffer 121 (noatstep ST642). 

75 FIG. 57 is a flowchart for explaining another exam- 
ple of parallel-processing of the pack-tfsaesembling of 
sub-picture data stream and decoding thereof, which 
data stream is encoded according to the process of FIG. 
53. 

so The process of FIG. 54 is parallel processing 
wherein the disassembling of the sLb-picture packs and 
the decoding thereof are independent from each other 
with respect to time. On the other hand, the process of 
FIG. 57 is other paraOel processing wherein the disas- 

25 sembfing of the sub-picture pacta and the decoding 
thereof are linked together with respect to time More 
specifically, in the emb odim en t of FIG. 57, the disas- 
sembling of the sub-picture packs and the decoding 
thereof are performed with substantially the same 

so speed. 

In the process of FIG. 57, the decocfing system 
reads the ID of the transferred data stream, so that only 
selected sub-picture packs (separated from the data 
stream) are sent to the sub-picture decoder (eg., sub- 
as picture decoder 101 in FIG. 1 1 or FIG. 17) (step ST40). 

When the first pack transmission is performed, 
index parameter "I" is set to "1" (step ST42), and disas- 
sembling of the first sub-picture pack is executed (step 
SST44). 

40 The disassembled pack is temporarily stored in a 
sub-picture buffer (memory 108) (step ST46). Thereaf- 
ter, index parameter V is set to index parameter y 
(step ST48), and index parameter T is incremented by 
"1"(6tepST50). 

4s When the incremented i-th pack exists, or the pack 
disassembled at preceding step ST44 is not the final 
pack (no at step ST52), the disassembling for the incre- 
mented i-th sub-picture pack is executed (step ST44). 
The disassembled i-th sub-picture pack (e.g. ( the 

so second pack) is temporarily stored in toe sub-picture 
buffer (or memory 108) as in the case of the first disas- 
sembled pack (step ST46), and index parameter V is 
further Incr em ented by "1" (step ST50). 

As mentioned above, a plurality of sub-picture 

ss packs are continuously disassembled (step ST44) while 
index parameter T is sequentially incremented, and 
disassembled packs are stored In the sub-picture buffer 
(or memory 109) (step ST46). 
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When continuously incremented i-th pack is not 
found, or the pack disassembled at preceding step 
ST44 is the final pack (yes at step ST52), the sub-pic- 
ture pack disassembling process at the stream to be 
decoded is ended. s 

During the continuous execution of cisassembling 
the sub-picture packs (steps ST44 to ST52), decoding 
of the sub-picture pack temporarily stored in the sub- 
picture buffer (memory 108) is performed in parallel to 
the disassembling of the sub-picture packs. w 

More specifically; when index parameter Tie set to 
index parameter "j" (step ST48), the first 0=1) sufc>pic- 
ture pack is read from memory 108 (step ST62), and the 
first Q=1) sib-picture pack is decoded (step STC4). 

During the decoding process (steps ST62 to ST64) is 
of the t-th Q=1) sub-picture pack, the cisassembling 
process (step ST44) of the i-th (i=2; incremented at step 
ST50) sub-picture pack is parallel executed. 

The above decoding process (steps ST62 to STC4) 
is repeated unless end command CMD_END is exe- so 
cutBd (no at step ST66). In this embodiment the decod- 
ing process is terminated by the execution of end 
command CMD_END (yes at step ST66). 

As has been described above, according to the 
present invention, wasteful sub-picture data in respect 2s 
of display space and display time can be greatly 
reduced. At the same time, the degree of freedom in 
sub-picture expressions equivalent to the bit map data 
scheme can be achieved, and a variety of applications 
of the sub-pictures can be assured. 30 

That is. according to the present invention, the use 
range setting information for setting the range to be 
used for a display in the sub-picture content data is 
recorded not to display data failing outside the use 
range. Wasteful data In respect of display space, which ss 
are produced in transmitting one-frame data to the dis- 
play system, can be greatly reduced. 

In addition, according to the present mention, the 
color setting information and the mixing ratio setting 
information for each pixel type of sub-picture content 
data, such as a pattern pixel, an outline, and a back- 
ground, are recorded to obtain sub-picture content data 
as information representing the shape of a sub-picture 
image. A sub-picture shape expression almost equal to 
the conventional scheme for providing color information 45 
and mixing ratio information for each pixel can be guar- 
anteed in a smaller volume of data. 

Furthermore, according to the present invention, 
the color/mixing ratio change setting information for set- 
ting, in units of pixels, change in the color of the sub-pic- so 
ture content data for each pixel type and in the mixing 
ratio of the main picture and the &jb-picture content 
data for each pixel type is recorded. A sub-picture can 
be dynamicalycfisplayed with an accuracy equal 10 that 
of the conventional bit map data scheme, but in a ss 
smaller volume ol data than that 

The color information of a sub-picture rarely 
changes in units of pixels, and the data volume of the 



color/mixing ratio change setting information is not 
expected to increase excessively. 

In addition, according to the present invention, even 
if the color of a sub-picture image changes, the sub-pic- 
ture can be displayed across a plurality of frame times 
using the same sub-picture content data as long as the 
shape does not change. As compared with the conven- 
tional scheme wherein the sub-picture data is kept sup- 
plied to the display system at the frame period 
regardless of changes in oofor and shape of the sub-pic- 
ture, wasteful sub-picture data in respect of display 
space can be greatly reduced. 

Claims 

1. A method of recording data wherein a sub-picture 
simultaneously reproducible with a main picture is 
converted into one or more sub-picture packets 
which are recorded to a recording medium, said 
method comprising the steps of: 

preparing packet header information (Phi) 
containing a playback start time of the sub-picture 
packets, raid playback start time being determined 
based on a given reference time; 

preparing sub-picture data (PXD) represent- 
ing a content of the sub-picture to be displayed: 

preparing one or more display control 
sequence information pieces (DCSQT) which indi- 
cate an order of display control for the content of 
said sub-picture data (PXD); 

preparing sub-picture header information 
(SPUH) containing information of a size of the 
packet of said sii>-picture data (PXD) and informa- 
tion of a recorded position of said display control 
sequence information pieces (DCSQT); and 

preparing each packet of the sub-picture in 
accordance with at least said prepared packet 
header information (PH), said prepared sub-picture 
data (PXD), said prepared display control 
sequence information pieces (DCSQT), and said 
prepared sub-picture header information (SPUH). 

2. A method of recording data wherein a sub-picture 
simultaneously reproducible with a main picture is 
converted into one or more sub-picture packets 
which are recorded, as one sub-picture data block, 
to a recoroing medium, said method comprising the 
steps of: 

preparing packet header information (PH) 
containing a playback start time of the sub-picture 
packets, said playback start time being determined 
based on a given reference time; 

preparing sub-picture data (PXD) represent- 
ing a content of the sub-picture to be displayed; 

preparing one or more display control 
sequence information pieces (DCSQT) which indi- 
cate an order of display control for the content of 
said sii>picture data (PXD); 

preparing sub-picture header information 
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(SPUH) containing information of a size of the 
packet of said sub-picture data (PXD) and informa- 
tion of a recorded position of said display control 
sequence information pieces (DCSQT); 

preparing respective packets of the sub-pic- 
ture in accordance with at least said prepared 
packet header information (PH), said prepared sub- 
picture data (PXD), said prepared display control 
sequence information pieces (DCSQT), and said 
prepared sub-picture header information (SPUH); 
and 

preparing playback time information (PTS) 
for the sub-picture packets in the sub-picture data 
block, and recording the playback time information 
(PTS) only to a frst one of the sub-picture packets. 

3. The method of claim 1 or 2, characterized in that 
one (DCSQ of FIG. 33) of said display control 
sequence information pieces (DCSQT) includes: 

display start and display stop time informa- 
tion (SPDCTS) of the sub-picture; 

record position information (SPNDCSQA) of 
the sub-picture data to be displayed; and 

a group of display control commands 
(SPDCCMD1. SPDCCMD2....) for the sub-picture 
data located at a position indicated by said record 
position information (SPNDCSQA). 

4. The method of claim 3. characterized in that said 
display start and display slop time information 
(SPDCTS) indicates display start time of the sub- 
picture as well as cfisplay stop time of the sub-pic- 
ture, which are defined as relative time with respect 
to the playback start time of said sub-picture pack- 



5. The method of claim 3, characterized in that display 
start control information (STA_DSP in FIG. 34) for 
controlling start of cfisplay of said sub-picture data 
based on the display start time information included 
in said display control sequence information pieces 
(DCSQT) is recorded as said display control com- 
mands (SPDCCMD1, SPDCCMD2,...). 

6. The method of claim 3. characterized in that display 
stop control information (STP_DSP in FIG. 34) for 
controlling stop of display of said sub-picture data 
based on the display stop time Information included 
in said display control sequence information pieces 
(DCSQT) is recorded as said display control com- 
mands (SPDCCMD1, SPDCCMD2,...). 

7. The method of claim 3, characterized In that color 
set information (SET_COLDR in FIG. 34) for setting 
color of respective types of pixels constituting said 
sub-picture data is recoided as said display control 
commands (SPDCCMD1, SPDCCMD2,...). 



8. The method of claim 3, characterized in that mix- 
ture ratio set information (SET_CONTR in FIG. 34) 
for setting a mixture ratio of the sub-picture data 
with respect to the main picture is recoided as said 

5 cfisplay control commands (SPDCCMD1. 
SPDCCMD2,...). 

9. The method of daim 3, characterized in that display 
area set Information (SET_DAREA in FIG. 34) tor 

to setting a display area of the sub-picture data on 
sad main picture is recorded as said cfisplay control 
commands (SPDCCMD1 , SPDCCMD2....). 

10. The method of daim 3, characterized in thai use 
75 area set information (SET_DSPXA in FIG. 34) for 

setting an area to be used tor displaying a content 
of the sub-picture data is recorded as sad display 
control commands (SPDCCMD1. SPDCCMD2,...). 

20 11. The method of claim 3. characterized in that 
color/mixture ratio change set information 
(CHG.COLCON in FIG. 34) for setting, in unit of 
pixel, a change in color of respective pixel types of 
the sub-picture data, and for setting, in unit of pixel, 

25 a change in a mixture ratio ol the sub-picture data 
with respect to the main picture, is recorded bs said 
display control commands (SPDCCMD1, 
SPDCCMD2,...). 

so 12. The method of daim 3, characterized in that display 
control end set information (CMD.END in FIG. 34) 
for setting a termination of display control of the 
sti>picture data is recoided as said cfisplay control 
commands (SPDCCMD1, SPDCCMD2,...)- 

38 

13. A recording mediwi for recording one or more sub- 
picture packets containing a sub-picture simultane- 
ously reproducible with a main picture, said record- 
ing medium containing: 

40 packet header information (PH) corrtariing a 

playback start time of the sub-picture packets, said 
playback start time being determined based on a 
given reference time; 

sub-picture data (PXD) representing a corv 

45 tent of the sub-picture to be displayed; 

one or more display control sequence infor- 
mation pieces (DCSQT) which indicate an order of 
cfisplay control for the content of said sub-picture 
data (PXD); and 

so sub-picture header Information (SPUH) con- 

taining information of a size of the packet of said 
sub-picture data (PXD) and information of a 
recorded position of said display control sequence 
information pieces (DCSQT), 

ss wherein each packet of the sii>-picture is 

prepared in accordance with at least said packet 
header information (PH), said sub-picture data 
(PXD), said display control sequence information 
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pieces (DCSQT), and said sub-picture header infor- 
mation (SPUH). 

14. A recording medium for recoixfing, in unit of a sub- 
picture data block, one or more sub-picture packets 
containing a sub-picture simultaneously reproduci- 
ble with a main picture, said recording medium con- 
taining: 

packet header information (PH) containing a 
playback start time of the sub-picture packets, said 
playback start time being determined based on a 
given reference time; 

sub-picture data (PXD) representing a con- 
tent of the sub-picture to be displayed; 

one or more cfispiay control sequence infor- 
mation pieces (DCSQT) which indicate an order of 
display control for the content of said sub-picture 
data (PXD); 

sub-picture header information (SPUH) con- 
taining information of a size of the packet of said 
sub-picture data (PXD) and information of a 
recorded position of said display control sequence 
information pieces (DCSQT); and 

playback time information (PTS) for deter- 
mining playback time of the sub-picture packets in 
said sub-pichra data block, 

wherein respective packets of the sub-pic- 
ture are prepared in accordance with at least said 
packet header information (PH), said sub-picture 
data (PXD), said display control sequence informa- 
tion pieces (DCSQT), and said sub-picture header 
information (SPUH), and 

said playback time information (PTS) is 
recorded only to a first one of the sub-picture pack- 
ets in said sub-picture data block. 

15. A method of encoding image information wherein a 
sub-picture simultaneously reproducible with a 
main picture is encoded into one or more sub-pic- 
ture packets, said method comprising the steps of: 

preparing packet header information (PH in 
FIG. 3; 3 in FIG. 29) containing a time stamp (PTS 
in FIG. 28 or 30) which indicates a playback start 
time of the sub-picture packets; 

preparing sii>picture information (32 in FIG. 
3 or 29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according to a 
predetermined manner; 

preparing display control sequence informa- 
tion (33 In FIG. 3 or 29) containing one or more cfis- 
play control sequence info r matio n pieces (DCSQT 
in FIG. 3 or 29; SPDCSQT in FIG. 32) which control 
an order of display of the sub-pi Aire based on said 
subi>»cture information (32); 

preparing sub-picture header information 
(31 in FIG. 3 or 29; SPUH in FIG. 3 or 31) contain- 
ing size information (SPDSZ) of the sub-picture 
packet and location Information (SPDCSQTA) of 
mid display control sequence information (33) ; and 



encoding each packet of the sub-picture in 
accordance with said prepared packet header infor- 
mation (3), said prepared sub-picture information 
(32), said prepared display control sequence infor- 
5 matton (33), and said prepared sub-picture header 
information (31). 

16. A method of encoding image information wherein a 
sub-picture simultaneously reproducible with a 

w main picture is encoded into two or more sub-pic- 
ture packets, 

(a) wherein, as for a first one of the sub-pictm 
packets, said method comprises the steps of: 

is preparing packet header information 

(PH in FIG. 3; 3 in FIG. 29) containing a time 
stamp (PTS in FIG. 28 or 30) which indicates a 
playback start time of the sub-picture packets; 
preparing sub-picture information (32 in 

20 FIG. 3 or 29} containing pixel data (PXD) con- 

stituting the sub-picture and being compressed 
according to a predetermined manner; 

preparing display control sequence 
information (33 in FIG. 3 or 29) containing one 

25 or more display control sequence information 

pieces (DCSQT in FIG. 3 or 29; SPDCSQT in 
FIG. 32) which control an order of display of the 
sub-picture based on said sub-picture informa- 
tion (32); and 

30 preparing sub-picture header informa- 

tion (31 in FIG. 3 or 29; SPUH in FIG. 3 or 31) 
containing size information (SPDSZ) of the 
sub-picture pactet and location Information 
(SPDCSQTA) of said display control sequence 

35 information (33), and 

(b) wherein, as for a second and subsequent 
ones of the sub-picture packets, said method 
comprises the steps of: 

preparing sub-picture i nformatio n (32 in 
40 FIG. 3 or 29) containing pixel data (PXD) con- 

stituting the sub-picture and being compressed 
acconefing to a predetermined manner; 

preparing display control sequence 
information (33 in FIG. 3 or 29) containing one 
45 or more display control sequence Information 

pieces (DCSQT In FIG. 3 or 29; SPDCSQT in 
FIG. 32) which control an order of display of the 
sub-picture based on said sii>-picture informa- 
tion 02); and 

so preparing sii>-picture header informa- 

tion (31 in FIG. 3 or 29; SPUH in FIG. 3 or 31) 
containing size information (SPDSZ) of the 
sub-picture pactet and location information 
(SPDCSQTA) of said display control sequence 

ss information (33). 

17. The method of claim 15 or 16. characterized in that 
said display control sequence information (33) 
includes: 
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time data (SPDCTS) tor specifying display 
start time and display stop time of the sub-pictire; 

address data (SPNDCSQA) for specifying a 
position of the sub-picture to be displayed; 

a command group of display control com- 
mands (SPDCCMD1, SPDCCMD2,...) for control- 
ling display of the sub-picture at the position 
specified by said address data (SPNDCSQA). 

18. The method of claim 1 5 or 16, characterized in that 

one frame of the sub-picture information 
(32/PXD) comprises either a single field or a dual 
fields formed of top and bottom fields; 

one frame of the sifepicture information 
(32/PXD) is (fended into and located at the top field 
(61 in Fia 58) and the bottom field (62 in FI& 58), 
when an interlaced display Is performed for the sub- 
picture information (32/PXD); and 

one frame of the sii>-pjctuiB information 
(32/PXD) is located at the single field, when a non- 
interlaced display is performed for the sub-picture 
information (32/PXD). 

19. The method of claim 17, characterized in that said 
time data (SPDCTS) is defined as relative time with 
respect to the playback start time of said sub-pic- 
ture packets. 

20. The method of claim 17. characterized in that one 
display control command (SPDCMD) of said com- 
mand groip includes display start control informa- 
tion (STA_DSP) for controlling start of display of 
said sub-picture information (32/PXD) based an the 
time data (SPDCTS) included in said display con- 
trol sequence information pieces (DCSQT). 

21. The method of dam 17, characterized in that one 
display control command (SPDCMD) of said com- 
mand group includes (fisplay stop control informa- 
tion (STP_DSP) for controlling stop of display of 
said sub-picture information (32/PXD) based an the 
time data (SPDCTS) included in said display con- 
trol sequence information pieces (DCSQT). 

22. The method of claim 17, characterized in that one 
display control command (SPDCMD) of said com- 
mand grotp includes color set information 
(SETCOL0R) for setting color of pixels (PXD) 
constituting said sub-picture information (32). 

23. The method of daim 17, characterized in that one 
display control command (SPDCMD) of said com- 
mand group includes mixture ratio set information 
(SETjCONTR) for setting a mixture ratio of the 
sub-picture with respect to the main picture. 

24. The method of claim 17, characterized in that one 
display control command (SPDCMD) of said com- 
mand group includes display area set information 



(SET_DAREA) for setting a display area of the sub- 
picture on the main picture. 

25u The method of claim 17, characterized in that one 
5 display control command (SPDCMD) of said com- 
mand group includes use area set information 
(SET.DSPXA) for setting an area to be used for 
displaying a content (PXD) of the sub-picture in a 
display area of the sib-picture. 

10 

26. The method of claim 17, characterized in that one 
display control command (SPDCMD) of said com- 
mand group includes color/contrast switch control 
information (CHGLCOLCON) for setting, in urat of 

75 pixel (PXD) constituting the sub-picture information 
(32), at least one of: 

a change in color of respective pixels of the 
sub-picture information (32), and 

a change in contrast of the sub-picture with 
20 respect to the main picture. 

27. The method of claim 17. characterized in that one 
display control c om mand (SPDCMD) of said com- 
mand group includes display control end command 

25 (CMD_END) for setting a termination of display 
control of the sub-picture. 

28. A recording medium for recording one or more si4>- 
picture packets containing a sub-picture simuttane- 

eo ously reproducible with a main picture, 6aid record- 
ing medium containing: 

packet header information (PH in FIG. 3; 3 in 
FIG. 29) containing a time stamp (PTS in FIG. 28 or 
30) which indicates a playback start time of the sub- 
as picture packets; 

sub-picture header information (31 in FIG. 3 
or 29; SPUH in FIG. 3 or 31) containing size infor- 
mation (SPDSZ) of the sub-picture packet 

sub-picture information 02 in FIG. 3 or 29) 
4o containing pixel data (PXD) constituting the sub- 
picture and being compressed according to a pre- 
determined manner; and 

display control sequence information (33 in 
FIG. 3 or 29) containing one or more display control 
45 sequence information pieces (DCSQT in FIG. 3 or 
29; SPDCSQT in Fia 32) which control an order of 
display of the sub-picture based on said sub-picture 
information (32). 

so 29. A recording medium for recordkig two or more sub- 
picture packets each containing a sub-picture 
simultaneously reproductole with a main picture, 

(a) wherein, as for a foist one of the sub-picture 
ss packets, said recording mediun contains: 

packet header information (PH in FIG. 3; 
3 in FIG. 29) containing a time stamp (PTS in 
FIG. 28 or 30) which indicates a playback start 
time of the sub-picture packets; 
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sub-picture header information (31 in 
FIG. 3 or 29; SPUH in FIG. 3 or 31) containing 
size information (SPDSZ) of the sub-picture 
packet; 

sub-picture information (32 in FIG. 3 or s 
29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according 
to a predetermined manner; and 

tf splay control sequence in forma tion 
(33 in FIGL 3 or 20) containing one or more dis- 10 
play control sequence information pieces 
(DCSQT in FIG. 3 or 29; SPDCSQT in FIG. 32) 
which control an order of display of the sub-pic- 
ture based on said sub-picture information 
(32), and 1S 
(b) wherein, as for a second and subsequent 
ones of the sub-picture packets, said recording 
medium contains* 

sub-picture header information (31 in 
FIG. 3 or 29; SPUH in FIG. 3 or 31) containing 20 
size information (SPDSZ) of the sub-picture 
packet; 

sub-picture information (32 in FIG. 3 or 
29) containing pixel data (PXD) constituting the 
sub-picture and being compressed acoonding 25 
to a predetermined manner; and 

display control sequence information 
(33 in FIG. 3 or 29) containing one or more dis- 
play control sequence information pieces 
(DCSQT in FIG. 3 or 29; SPDCSQT in FIG. 32) so 
which control an order of display of the sub-pic- 
ture based on said sub-picture information 
(32). 

30. A method of encoding image information wherein a as 
sub-picture simultaneously reproducible with a 
main picture is encoded into one or more sub-pic- 
ture packets, said method comprising the steps of: 

preparing packet header information (PH in 
FIG. 3; 3 in FIG. 29) containing a time stamp (PTS 40 
in FIG. 28 or 30) which indicates a playback start 
time off the sub-picture packets; 

preparing sub-picture header information 
(31 in FIG 3 or 29; SPUH in FIG. 3 or 31) contain- 
ing size information (SPDSZ) of the sub-picture 45 
packet; 

preparing sub-picture information (32 in FIG. 
3 or 29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according to a 
predetermined manner; and so 

preparing display control sequence informa- 
tion (33 in FIG. 3 or 29) containing one or more dis- 
play control sequence information pieces (DCSQT 
in FIG 3 or 29; SPDCSQT in FIG. 32) which control 
an order of display of the sub-picture based on said ss 
sub-picture information (32). 

31. A method of encoding image information wherein a 
sub-picture simultaneously reproducible wrth a 



main picture is encoded into two or more sub-pic- 
ture packets, said method comprising the steps of: 

(a) wherein, as for a first one of the sub-picture 
packets, said method comprises the steps of: 

preparing packet header information 
(PH in Fia 3; 3 in FIG. 29) containing a time 
stamp (PTS in FIGL 28 or 30) which indicates a 
playback start time of the sub-picture packets; 

preparing sub-picture header informa- 
tion (31 in FIG. 3 or 29; SPUH in FIG. 3 or 31) 
containing size kiformation (SPDSZ) of the 
sub-picture packet; 

preparing sub-picture information (32 in 
FIG. 3 or 29) containing pixel data (PXD) con- 
stituting the sub-picture and being compressed 
according to a predetermined manner; and 

preparing display control sequence 
information (33 in FIG. 3 or 29) containing one 
or more display control sequence information 
pieces (DCSQT in FIG. 3 or 29; SPDCSQT in 
FIG. 32) which control an order of display of the 
sub-picture based on said sub-picture informa- 
tion (32). and 

(b) wherein, as for a second and subsequent 
ones of the sub-picture packets, said method 
comprises the steps of: 

preparing sub-picture header informa- 
tion (31 in FIG. 3 or 29; SPUH in FIG. 3 or 31) 
containing size information (SPDSZ) of the 
sub-picture packet; 

preparing sub-picture in for m atio n 02 in 
FIG. 3 or 29) containing pixel data (PXD) con- 
stituting the suthpicture and being compressed 
according to a predetermined manner; and 

preparing display control sequence 
information (33 in FIG. 3 or 29) containing one 
or more display control sequence information 
pieces (DCSQT in FIG. 3 or 29; SPDCSQT in 
FIG. 32) which control an order off display of the 
sub-ptatira based on said sub-picture informa- 
tion (32). 

A method of encoding image information wherein a 
sub-picture simultaneously reproducible with a 
main picture Is encoded in unit of a packet, said 
method comprising the steps of: 

converting (ST10) a content of tfie sii>-p8c- 
ture into bit-mapped sub-picture data; 

preparing (ST12) a spatial parameter 
(SET_DAREA) for setting a cfsplay area of the sub- 
picture on a display area of the main picture; 

compressing (ST14) the bit-mapped sub- 
picture data based on a predetermined manner so 
as to generate compressed pixel data (PXD); 

preparing a time parameter 
(STA_DSP/STP_DSP) for specifying display start 
timing and dspiay stop timing of the sub-picture 
(ST16); 
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preparing (ST18) one or more display con- 
trol sequence table (DCSQT) for controlling an 
order of display of the sub-pbture; 

preparing (ST20) a sub-picture unit header 
(SPUH) containing size information (SPDSZ) of a 5 
packet of the cub-picture and location information 
(SPDCSQTA) of the display control sequence table 
(DCSQT), and combining the conrpressed pixel 
data (PXD) and the display control sequence table 
(DCSQT) with the sii^ptcture unit header (SPUH) w 
60 as to provide a sub-picture unit (30); 

making (ST24) a packet of the sub-picture 
unit (30) when a size of the sub-picture unit (30) 
exceeds a predetermined value (2k bytes); 

recording (ST26) a time stamp (PTS) at the 75 
packet of the sub-picture unit (30) having a size 
exceeding the predetermined value (2k bytes), or at 
the sub-picture unit (30) having a size not more 
than the predetermined value (2k bytes), said time 
stamp (PTS) representing a playback start time of & 
the subhpicture; and 

packing (ST28) the packet of said sub-pic- 
ture unit (30) having the time s tamp (PTS), or pack- 
ing (ST28) said sub-picture unit (30) having the 
time stamp (PTS), so as to make a data stream 25 
containing data of the packed sub-picture unit (30) 
and other data. 

33. An encodingAdecoding system wherein a sii>-pic- 
ture simultaneously reproducfele with a main pic- so 
ture is encoded in unit of a packet, said system 
comprising the steps of: 

converting (ST10) a content of the sub-pic- 
ture into bH-mapped sub-picture data; 

preparing (ST12) a spatial parameter as 
(SETJDAREA) for setting a display area of the sub- 
picture on a display area of the main picture; 

compressing (ST14) the bit-mapped sub- 
picture data based on a predetermined manner so 
as to generate compressed pixel data (PXD); 40 

preparing a time parameter 
(STA_DSP/STP_DSP) tor specifying display start 
timing and display stop timing of the sub-picture 
(ST16); 

preparing (ST18) one or more display con- 45 
trol sequence table (DCSQT) for contrallkig an 
order of display of the sub-picture; 

preparing (ST20) a sub-picture unit header 
(SPUH) containing size information (SPDSZ) of a 
pacta! of the sub-picture and location info rma tion so 
(SPDCSQTA) of the display control sequence table 
(DCSQT), and combining the conrpressed pixel 
data (PXD) and the di^jJay control sequence table 
(DCSQT) with the sub-picture unit header (SPUH) 
so as to provide a sub-picture unit (30); ss 

making (ST24) a packet of the sub-picture 
unit (30) when a size of the sii>pictiire unit (30) 
exceeds a predetermined value (2k bytes); 

recording (ST26) a time 6tamp (PTS) at the 



packet of the sub-picture unit (30) having a size 
exceeding the predetermined value (2k bytes), or at 
the sii>picture unit (30) having a size not more 
than the predetermined value (2k bytes), said time 
stamp (PTS) representing a playback start time of 
the sub-picture; 

packing (ST28) the packet of said sub-pic- 
lure unit (30) having the time stamp (PTS), or pack- 
ing (S128) said sub-picture unit (30) having the 
lime stamp (PTS). so as to makB a data stream 
containing data of the packed sub-picture unit (30) 
and other data; 

separating (ST40) the data of the packed 
sub-picture unit (30) from the data stream; 

disassembling (ST44) the pack of the sepa- 
rated sub-picture unit (30); 

extracting (ST448) the time stamp (PTS) 
from the disassembled pack when the time stamp 
(PTS) is recorded at the disassembled pack; and 

conparing (ST640) a content of the 
extracted time stamp (PTS) with time given by a 
system timer (120 in FIG. 11), checking (ST644) 
one or more contents of the display control 
sequence table (DCSQT) when the content of the 
extracted time stamp (PTS) matches the time given 
by the system timer (120), and expanding (ST646) 
the compressed pixel data (PXD) according to the 
checked contents of the cSsptay control sequence 
table (DCSQT). 

34. A method of decoding a data stream obtained by 
packing a plurality of sub-picture units (30) each of 
which includes: 

a lime stamp (PTS) i repeating a playback 
start time of bit-mapped sub-picture packets; 

display control sequence table (DCSQT) for 
controlling an order of cf splay of the sub-picture; 

pixel data (PXD) obtained by compressing 
the bit-mapped sub-picture; and 

sub-picture unit header (SPUH) containing 
size information (SPDSZ) of the sub-picture packet 
and location information (SPDCSQTA) of the dis- 
play control sequence table (DCSQT), said method 
comprising the steps of: 

separating (ST40) the data of the packed 
sife-picture unit (30) from the data stream; 

disassembling (ST44) the pack of the sepa- 
rated SLb-picture unit (30); 

extracting (ST44S) the time stamp (PTS) 
from the disassembled pack when the time stamp 
(PTS) is reoonded at the disassembled pack; and 

comparing (ST640) a content of the 
extracted time stamp (PTS) with time given by a 
system timer (120 in FIG. 11), checking (ST644) 
one or more contents of the display control 
sequence table (DCSQT) when the content of the 
extracted time stamp (PTS) matches the time given 
by the system timer (120), and expanding (ST646) 
the compressed pixel data (PXD) according to the 
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checked contents of the display control sequence 
table (DCSQT). 

35. An apparatus of encocfng image information 
wherein a sub-picture simultaneously reproducible s 
with a main picture is encoded into one or more 
sub-picture packets, said apparatus comprising: 

means (ST26) for preparing packet header 
information (PH/3) containing a time stamp (PTS) 
which indicates a playback start time of the sub-pic- io 
ture packets; 

means (ST14) for preparing sub-picture 
information (32) which contains pixel data (PXD) 
constituting the sub-picture and being compressed 
according to a predetermined manner; is 

means (ST12, ST16, ST 18) for preparing 
display control sequence information (33) contain- 
ing one or more display control sequence informa- 
tion pieces (DCSQT7SPDCSQT) which control an 
older of display of the 6ub-picture using said sub- 20 
picture information (32); and 

means (ST20) for generating each unit of the 
sub-picture packets in accordance with said packet 
header information (3), raid sub-picture information 

(32) , said display control sequence information 2s 

(33) , and sii>-picture header information (31) which 
contains size information (SPDSZ) of the sub-pic- 
ture packet and location information (SPDCSQTA) 
of the cfisptay control sequence information (33). 

30 

36. An apparatus of encoding image information 
wherein a sub-picture simultaneously reproductole 
with a main picture is encoded into two or more 
sub-picture packets, said apparatus comprising: 

35 

(a) first means (ST10-ST26) for encoding a first 
one of the sub-picture packets in accordance 
with: 

packet header information (PH in FIG. 3; 
3 in FIG. 29) containing a time stamp (PTS in 40 
FIG. 28 or 30) which indicates a playback start 
time of the sub-picture packets; 

sub-picture information (32 in FIG. 3 or 
29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according 4S 
to a predetermined manner; 

display control sequence information 
(33 in FIG. 3 or 29) containing one or more dis- 
play control sequence information pieces 
(DCSQT in FIG. 3 or 29; SPDCSQT in FIG. 32) bo 
which control an oder of display of the sub-pic- 
ture based on 6aid sub-picture in formatio n 

(32) ; and 

sub-picture header information (31 in 
FIG.3cr29;SPUHinFIG.3or31)oofitainlng ss 
size information (SPDSZ) of the sub-picture 
packet and location in fo rmation (SPDCSQTA) 
of raid display control sequence information 

(33) . and 



(b) second means (ST10-ST20) for encoding 
second and subsequent ones of the sub-pic- 
ture packets In accordance with: 

sub-picture information (32 in FIG. 3 or 
29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according 
to a predetermined manner; 

display control sequence information 
(33 in FIG. 3 or 29) containing one or more dis- 
play control sequence Information pieces 
(DCSQT in FIG. 3 or 29; SPDCSQT in FIG 32) 
which control an order of display of the sib-pic- 
ture based on said sub-picture information 

(32) ; and 

sub-picture header information (31 in 
FIG. 3 or 29; SPUH in FIG. 3 or 31) containing 
size information (SPDSZ) of the sub-picture 
packet and location information (SPDCSQTA) 
of said display control sequence information 

(33) . 

37. A broadcasting system for encoding and broadcast- 
ing image information wherein a sub-picture simul- 
taneously reproducible with a maki picture is 
encoded into one or more sub-picture packets, said 
broadcasting system comprising: 

means (ST26) for preparing packet header 
information (PH/3) containing a time stamp (PTS) 
which indicates a playback start time of the sii>-pic- 
ture packets; 

means (ST14) for preparing sub-picture 
information (32) which contains pixel data (PXD) 
constituting the sii>-picture and being compressed 
according to a predetermined manner; 

means (ST12, ST16 P ST18) for preparing 
display control sequence information (33) contain- 
ing one or more display control sequence informa- 
tion pieces (DCSQT/SPDCSQT) which control an 
order of cfisplay of the sub-picture using said sub- 
picture information (32); 

means (ST2D) for generating each unit of the 
sub-picture packets in accordance with said packet 
header information (3). said sub-picture information 
02), said display control sequence information 
(33). and sub-picture header information (31) which 
contains size information (SPDSZ) of the sub-pic- 
ture packet and location information (SPDCSQTA) 
of the display control sequence information (33); 
and 

means (FIG. 22) for broadcasting a data 
stream containing the generated sub-picture pack- 
eta 

3a A broadcasting system for encoding and broadcast- 
ing image Information wherein a sub-picture simul- 
taneously reproducible with a main picture is 
encoded into two or more sub-picture packets, said 
broadcasting system comprising: 
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(a) first means (ST1 0-ST26) for encoding a first 
one of the sub-picture packets in accordance 
with: 

packet header information (PH in FIG. 3; 
S in FIG. 29) containing a time stamp (PTS in $ 
FIG. 28 or 30) which indicates a playback start 
time of the sii>-piciurB packets; 

sub-picture Information (32 in FIG. 3 or 
29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according w 
to a predetermined manner; 

di^lsy control sequence information 
(33 in FIG. 3 or 29) containing one or more dis- 
play control sequence information pieces 
(DCSQT ii FIG. 3 or 29; SPDCSQT in FIG. 32) 1S 
which control an order of display ol the sub-pic- 
ture based on said sub-picture information 

(32) ; and 

sub-picture header information (31 in 
FIG 3 or 29; SPUH in FIG. 3 or 31) containing so 
size information (SPDSZ) of the sub-picture 
packet and location information (SPDCSQTA) 
of said display control sequence information 

(33) . 

(b) second means (ST10-S720) fa encoding 25 
second and subsequent ones of the sub-pic- 
ture packets in accordance with: 

sub-picture information (32 in FIG. 3 or 
29) containing pixel data (PXD) constituting the 
sub-picture ami being compressed according so 
to a predetermined manner; 

display control sequence information 
(33 in FIG. 3 or 29) containing one or more dis- 
play control sequence information pieces 
(DCSQT in FIGL 3 or 29; SPDCSQT in FIG 32) as 
which control an order of display of the sub-pic- 
ture based on said sub-picture information 

(32) ; and 

sub-picture header information (31 in 
FIG 3 or 2$ SPUH in FIGL 3 or 31) containing 40 
size information (SPDSZ) of the sub-picture 
packet and location information (SPDCSQTA) 
of said display control sequence information 

(33) . and 

(c) third means (FIG 22) for broadcasting a 45 
data stream containing the sub-picture packets 
generated by either of said first and second 
means. 

39. An apparatus of encoding image infor ma t i on so 
wherein a sub-picture simultaneously reproducible 
with a main picture is encoded in unit of a packet, 
said apparatus conrprising: 

means for converting (ST10) a content of the 
sii>-picture into bit-mapped sub-picture data: ss 

means for preparing (ST12) a spatial param- 
eter (SET_DAREA) for setting a display area of the 
sub-picture on a display area erf the main picture; 

means for compressing (ST14) the bit- 



mapped sub-picture data based on a predeter- 
mined manner so as to generate compressed pixel 
data (PXD); 

means for preparing a time parameter 
(STAJ)SP/STP_DSP) for specifying display start 
timing and display stop timing of the sub-picture 
(ST16); 

means for preparing (ST1 8) one or more dis- 
play control sequence table (DCSQT) for controlEng 
an order of display of the sub-picture; 

means for preparing (ST20) a sub-picture 
unit header (SPUH) containing size information 
(SPDSZ) of a packet of the sub-picture and location 
information (SPDCSQTA) of the display control 
sequence table (DCSQT), and combining the com- 
pressed pixel data (PXD) and the display control 
sequence table (DCSQT) with the sub-picture unit 
head©* (SPUH) so as to provide a sub-ptctire unit 
(30); 

means for making (ST24) a packet of the 
sub-picture unit (30) when a size of the sub-picture 
unit (30) exceeds a predetermined value (2k bytes); 

means for recording (ST26) a time stamp 
(PTS) at the packet of thesi4>-picture unit (30) hav- 
ing a size exceeding the predetermined value (2k 
bytes), or at the sub-picture init (30) having a size 
not more than the predetermined value (2k bytes), 
said time stamp (PTS) representing a playback 
start time of the sub-picture; and 

means (1 12) for packing (ST28) the packet 
of said sLfc-piclLro unit (SO) having the time stanp 
(PTS), or packing (ST28) said sub-picture unit (30) 
having the time stamp (PTS}, so as to make a data 
stream containing data of the packed sub-picture 
unit (30) and other data. 

4a An encocfing/decoding apparatus wherein a sub- 
picture simultaneously reproducible with a main 
picture is encoded in init of a packet, said appara- 
tus comprising: 

means for converting (ST1 0) a content of the 
sub-picture into bit-mapped sub-picture data; 

means for preparing (ST12) a spatial param- 
eter (SET_DAREA) for setting a display area of the 
sub-picture on a display area of the main picture; 

means for compressing (ST14) the bit- 
mapped sub-picture data based on a predeter- 
mined manner so as to generate compressed pixel 
data (PXD); 

means for preparing a tine parameter 
(STA_DSP/STP_DSP) for specifying display start 
timing and display stop tinting of the sub-picture 
(ST16); 

means for preparing (ST1 8) one or more dis- 
play control sequence table (DCSQT) for controllng 
an order of display of the 6ul>picture; 

means for preparing (ST20) a sub-picture 
unit header (SPUH) containing size Information 
(SPDSZ) of a packet of the sub-picture and location 
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information (SPDCSQTA) of the display control 
sequence table (DCSQT), and combining the com- 
pressed pixel dale (PXD) and the display control 
sequence table (DCSQT) with the sub-picture unit 
header (SPUH) so as to provide a sub-picture unit $ 
(30); 

means for making (ST24) a packet of the 
sub-picture unit (30) when a size of the sub-picture 
unit (30) exceeds a predetermined value (2k bytes); 

means lor recording (ST26) a time stamp w 
(PTS) at the packet of the sub-picture unit (30) hav- 
ing a size exceeding the predetermined value (2k 
bytes), or at the sub-picture unit (30) having a size 
not more than the predetermined value (2k bytes), 
said time stamp (PTS) representing a playback 15 
start time of the sub-picture; 

means lor packtog (ST28) the packet of said 
sub-picture unit (30) having the fame stamp (PTS), 
or packing (ST28) said sub-picture unit (30) having 
the fene stamp (PTS), so as to make a data stream 20 
containing data of the packed sub-picture unit (30) 
and other data; 

means for separating (ST40) the data of the 
packed sub-picture unit (30) from the data stream; 

means for disassembling (ST44) the pack of 25 
the separated sub-picture unit (30); 

means for extracting (ST448) the time stamp 
(PTS) from the disassembled pack when the time 
stamp (PTS) is recorded at the disassembled pack; 
and 30 

means (112) for comparing (ST640) a con- 
tent of the extracted time stamp (PTS) with time 
given by a system timer (120 in FIG. 1 1), checking 
(ST644) one or more contents of the display control 
sequence table (DCSQT) when the content of the as 
extracted time stanp (PTS) matches the time given 
by the system timer (120), and expanding (ST646) 
the compressed (axel data (PXD) according to the 
checked contents of the display control sequence 
table (DCSQT). 40 

41. An apparatus of decotf ng a data stream obtained 
by packing a plurality of sub-picture units (30) each 
of which includes: 

a lime starry (PTS) incfcating a playback 4a 
start lime of bit-mapped sub-picture packets; 

cfsplay control sequence table (DCSQT) for 
controlling an order of display of the sub-picture; 

pixel data (PXD) obtained by compressing 
the bit-mapped sub-picture; and so 

sub-picture unit header (SPUH) containing 
size information (SPDSZ) of the sub-picture packet 
and location Information (SPDCSQTA) of the dis- 
play control sequence table (DCSQT), said appara- 
tus comprising: be 

means for separating (ST40) the data of the 
packed sub-picture unit (30) from the data stream; 

means tor disassembling (ST44) the pack of 
the separated sub-picture unit (30); 



means tor extracting (ST448) the time stamp 
(PTS) from the disassembled pack when the time 
stamp (PTS) is recorded at the disassembled pack; 
and 

means (112) for comparing (ST640) a con- 
tent of the extracted time stamp (PTS) with time 
given by a system timer (120 in FIG. 11), checking 
(ST644) one or more contents of the cisplay control 
sequence table (DCSQT) when the content of the 
extracted time stamp (PTS) matches the time given 
by the system timer (120), and expanding (ST646) 
the compressed pixel data (PXD) accorcfing to the 
checked contents of the display control sequence 
table (DCSQT) so as to provide non-encoded sub- 
picture information. 

An apparatus of encoding and recording image 
information wherein a sub-picture sinruitaneousty 
reproducible with a main picture is encoded in unit 
of a packet, said apparatus comprising: 

means tor converting (ST10) a content of the 
sub-picture into bit-mapped sub-picture data; 

means tor preparing (ST12) a spatial param- 
eter (SET_DAREA) tor setting a display area of the 
sub-picture on a display area of the main picture; 

means for compressing (ST14) the bit- 
mapped sub-picture data based on a predeter- 
mined manner so as to generate compressed pixel 
data (PXD); 

means tor preparing a time parameter 
(STA_DSP/STP_DSP) tor specifying display start 
timing and cfsplay stop timing of the sub-picture 
(ST16); 

means tor preparing (ST18) one or more cfis- 
play control sequence table (DCSQT) for controlling 
an order of display of the sub-picture; 

means for preparing (ST20) a sub-picture 
unit header (SPUH) containing size information 
(SPDSZ) of a packet of the sub-picture and location 
information (SPDCSQTA) of the tfsplay control 
sequence table (DCSQT), and combining the com- 
pressed pixel data (PXD) and the display control 
sequence table (DCSQT) with the sub-picture unit 
header (SPUH) so as to provide a sub-picture unit 
(30); 

means for making (ST24) a packet of the 
sub-picture unit (30) when a size of the sub-picture 
unit (30) exceeds a predetermined value (2k bytes); 

means for recortfng (ST26) a time stamp 
(PTS) at the packet of the sub-picture unit (30) hav- 
ing a size exceeding the predetermined value (2k 
bytes), or at the sub-picture unit 00) having a size 
not more than the predetermined value (2k bytes), 
said time stamp (PTS) representing a playback 
start time of the sub-picture; 

means (112) for packing (ST28) the packet 
of said sub-picture unit (30) having the time stamp 
(PTS), or packing (ST28) said sub-picture unit (30) 
having the time stamp (PTS), so as to make a data 
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stream containing data of the packed sub-picture 
unit (30) and other data; and 

means (702-704) for recording specific infor- 
mation on a recording medium (OD), said specific 
information containing the data stream of said 5 
packed sub-picture unit (30). 

43. An apparatus of decocfing and playing back a data 
stream obtained by packing a plurality of sub-pic- 
ture units (30) each of which includes: 

a time stamp (PTS) indicating a playback 
start time of bit-mapped sd>picture packets; 

cf splay control sequence table (DCSQT) for 
controlling an order of display of the sub-picture; 

pixel data (PXD) obtained by compressing 
the bit-mapped sUb-picture; and 

sub-picture unit header (SPUH) containing 
size information (SPDSZ) of the sub-picture packet 
and location information (SPDCSQTA) of the dis- 
play control sequence table (DCSQT), said appara- 
tus conprising: 

means for separating (ST40) the data of the 
packed sub-picture unit 00) from the data stream; 

means for disassembling (ST44) the pack of 
the separated sub-picture unit (30); 

means for extracting (ST44S) the time stamp 
(PTS) from the disassembled pack when the lime 
stamp (PTS) is recorded at the disassembled pack; 

means (1 12) for comparing (ST640) a cm- 
tent of the extracted time stamp (PTS) with time 
given by a system timer (120 in FIG. 11), checking 
(ST644) one or more contents of the display control 
sequence table (DCSQT) when the content of the 
extracted time stamp (PTS) matches the time given 
by the system timer (120), and expanding (STS46) 
the compressed pixel data (PXD) according to the 
checked contents of the display control sequence 
table (DCSQT) so as to provide non-encoded sub- 
picture information. 

44. A method of encoding image information wherein a 
sub-picture simultaneously reprodudWe with a 
main picture is encoded into one or more sub-pic- 
ture packets, said method comprising: 

a first step of preparing packet header infor- 
mation (PH in FIG. 3; 3 In FIG. 20) containing a time 
stamp (PTS in FIG. 28 or 30) which indicates a 
playback start time of the sub-picture packets; 

a second step of preparing sub-picture infor- 
mation (32 to FIG. 3 or 29} containing pixel data 
(PXD) constituting the sub-picture and being com- 
pressed accord ng to a predetermined manner; 

a third step of preparing display control 
sequence information (33 in FIG. 3 or 29) contain- 
ing one or more display control sequence Informa- 
tion pieces (DCSQT in FIG. 3 or 29; SPDCSQT in 
FIG. 32) which control an order of display of the 
siihpfcture based on said sub-picture information 
(32): 



a fourth step of preparing sub-picture header 
information (31 in FIG. 3 or 29; SPUH in FIG. 3 or 
31) containing size information (SPDSZ) of the sub- 
picture packet and location information (SPDC- 
SQTA) of said display control sequence Information 
(33); and 

a fifth step of encoding each packet of the 
sii>-picture in accordance with said prepared 
packet header information (3), said prepared sub- 
picture information (32), said prepared display con- 
trol sequence information (33), and said prepared 
sib-pictm header information (31), 

wherein said second step includes: 

a step (STB01) of specifying a data block of 
one compression unit in the compressed pixel data 
(PXD); and 

a step (STB06) of generating a compressed 
unit data block in accordance with: 

a coding header corresponding to a continu- 
ous number of same pixBl data in the compressed 
unit data block, 

continuous pixel nunfcer data indicating the 
continuous number of the same pixel data, and 

data representing the same pixel data In the 
compressed unit data block. 

45. A method of decotfng a data stream obtained by 
packing a plurality of sub-picture units (30) each of 
which includes: 

a time stamp (PTS) indicating a playback 
start time of bit-mapped sub-picture packets; 

display control sequence table (DCSQT) for 
controlling an order of display cf the sub-picture; 

pixel data (PXD) obtained by compressing 
the bit-mapped sub-picture; and 

sub-picture unit header (SPUH) containing 
size information (SPDSZ) of the sub-picture packet 
and location information (SPDCSQTA) of the dis- 
play control sequence table (DCSQT), said method 
comprising the steps of: 

separating (ST40) the data of the packed 
sub-picture unit (30) from the data stream; 

disassembling (ST44) the pack of the sepa- 
rated sub-picture unit (30); 

extracting (ST448) the time stamp (PTS) 
from the disassembled pack whan the time stamp 
(PTS) is recorded at the disassembled pack; 

comparing (ST640) a content of the 
extracted time stamp (PTS) with time given by a 
system timer (120 in FIG. 11). checking (ST644) 
one or more contents of the display control 
sequence table (DCSQT) when the content of the 
extracted time stamp (PTS) matches the time given 
by the system timer (120), and expanding (ST646) 
the compressed pixel data (PXD) according to the 
checked contents of the cfisplay control sequence 
table (DCSQT); 

extracting (ST1101-ST11Q9) a coding 
header from a data block of one compression unit in 
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the compressed pixel data (PXD); 

extracting (ST1110-ST1113) continuous 
P'kbI number data from the data block of said one 
compression unit in accordance with a content of 
the coding header; s 

determining (ST1 1 14) a content of uncom- 
pressed pixel data constituting the one compres- 
sion in it in accordance with a remaining portion of 
the data Hock of said one compression unit from 
which said coding header and said continuous pixel io 
number data are removed; and 

arranging (ST1115-ST111B) the content of 
said uncompressed pixel data by a number indi- 
cated by said continuous pixel number data so as to 
restore uncompressed pixel pattern of the one com- f S 
pression unit. 

46. A method off encoding image information wherein a 
sub-picture simultaneously reproducible with a 
main picture is encoded into one or more sub-pic- 20 
ture packets, said method comprising the steps of: 

preparing packet header information (PH in 
FIG. 3; 3 in FIG. 29) containing a time stamp (PTS 
in FIG. 28 or 30) which incScates a playback start 
time of the sub-picture packets; 2s 

preparing sife-picture information (32 in FIG. 
3 or 29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according to a 
predetermined manner; 

preparing display control sequence irrforma- bo 
tion (33 in FIG. 3 or 29) containing one or more dis- 
play control sequence information pieces (DCSQT 
in FIG. 3 or 29; SPDCSQT in FIG. 32) which control 
an order of display of the sub-picture based on said 
sub-picture information (32), sad display control ss 
sequence information pieces (DCSQT) including 
display control start time information (SPDCTS), 
address information (SPNDCSQA) of subsequent 
display control sequences, and one or more display 
control commands (SPDCCMD); 40 

preparing sub-picture header information 
(31 in FIG. 3 or 29; SPUH in FIG. 3 or 31) contain- 
ing size Information (SPDSZ) of the sub-picture 
packet and location information (SPDCSQTA) of 
said display control sequence information (33); and 45 

encoding each packet of the sdb-picture in 
accordance with said prepared packet header infor- 
mation (3), said prepared sub-picture Information 
(32), said prepared display control sequence infor- 
mation (33), and said prepared sub-picture header bo 
information (31). 

47. A method of encoding image information wherein a 
sub-picture simultaneously reproducible with a 
main picture is encoded Into one or more siib-pic- bs 
ture packets, said method comprising the steps of: 

preparing packet header information (PH in 
FIG. 3; 3 In FIG. 29) containing a time stamp (PTS 
in FIG. 28 or 30) which indicates a playback start 



time of the sub-picture packets; 

preparing sub-picture information (32 in FIG. 
Sor29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according to a 
predetermined manner; 

preparing display control sequence informa- 
tion (33 in FIG. 3 or 29) containing one or more dis- 
play control sequence information pieces (DCSQT 
in FIG. 3 or 29; SPDCSQT in Fia 32) which control 
an order of display of the sub-picture based on said 
sub-picture information (32) r said display control 
sequence information pieces (DCSQT) including 
display control start time information (SPDCTS), 
address information (SPNDCSQA) of subsequent 
display control sequences, and one or more display 
control commands (SPDCCMD); 

preparing sub-picture header information 
(31 in FIG 3 or 29; SPUH in FIG. 3 or 31) contain- 
ing size information (SPDSZ) of the sub-picture 
packBt and location information (SPDCSQTA) of 
said display control sequence information (33), said 
sub-picture header information (SPUH) including 
sub-picture size information (SPDSZ) and start 
address information (SPDCSQTA) of the display 
control sequence; and 

encoding each packet of the sub-picture in 
accordance with said prepared packet header infor- 
mation (3), said prepared sub-picture information 
(32), said prepared display control sequence infor- 
mation (33), and said prepared sub-picture header 
information (31). 

48. A method of encoding Image information wherein a 
sub-picture simultaneously reproducible with a 
main picture is encoded into one or more sub-pic- 
ture packets, said method comprising the steps at 

preparing packet header information (PH in 
FIG. 3; 3 in FIG. 29) containing a time stamp (PTS 
in FIG. 28 or 30) which indicates a playback start 
time of the sub-picture packets; 

preparing sub-picture information (32 in FIG. 
3 or 29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according to a 
predetermined manner; 

preparing display control sequence informa- 
tion (33 in FIG. 3 or 29) containing one or more cfis- 
play control sequence information pieces (DCSQT 
in FIG. 3 or 29; SPDCSQT In FIG. 32) which control 
an order of display of the sub-picture based on said 
sub-picture information (32); 

preparing sub-picture header information 
(31 m FIG. 3 or 29; SPUH in FIG. 3 or 31)contain- 
ing size information (SPDSZ) of the sub-picture 
packet and location information (SPDCSQTA) of 
said display control sequence information (33), said 
sub-picture header in forma tion (SPUH) including 
sub-picture size Information (SPDSZ) and start 
address Information (SPDCSQTA) of the display 
control sequence; and 
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encoding each packet of the sub-picture in 
accordance with said prepared packet header infor- 
mation (3), said prepared sii>-picture information 
(32), said prepared display control sequence infor- 
mation (33), and said prepared sub-picture header $ 
information (31). 

49. The method of claim 46, characterized in that said 
one or more display control commands (SPDC- 
CMD) include: io 

a command (STAJDSP) for setting display 
start timing of pbels corresponding to the pixBl data 
(PXD); 

a command (STP_DSP) for setting display 
stop timing of pixels corresponding to the pixel data 75 
(PXD); 

a command (SETJDAREA) tor setting a dis- 
play area of pixels corresponding to the pixel data 
(PXD): and 

a command (SET_DSPXA) for setting a dis- so 
play start address of pixels corresponding to the 
pixel data (PXD). 

50. The method of claim 48, characterized in that said 
one or more display control commands (SPDC- ss 
CMD) further include: 

a command (SET_COLOR) for setting a 
cotor code of the pixel data (PXD). 

51. The method of claim 50, characterized in that said so 
one or more display control commands (SPDC- 
CMD) further include: 

a command (SET_CONTR) for setting a 
contrast of the pixel data (PXD) with respect to the 
main picture. 

52. The method of daim 51, characterized in that said 
one or more display control commands (SPDC- 
CMD) further include: 

a command (CHG_OOLCON) for changing 40 
at least one of the color code of the pixel data 
(PXD) and the contrast of the pixel data (PXD) with 
respect to thB main picture. 

53. A method of encoding image information wherein a 4s 
si±>-picture, simultaneously reproducible with a 
main picture and to be displayed with an interlaced 
mode, is encoded into one or more sub-picture 
packets, said method comprising the steps of: 

preparing packet header information (3; PH) so 
containing a time stamp (PTS) which indicates a 
playback start time of the sub-picture packets; 

preparing first sub-picture information (61) 
containing pixel data (PXD) constituting the sub- 
picture and being compressed according to a pre- ss 
determined manner, said first sub-picture informa- 
tion (61) constituting a top field of one display frame 
of said sii>picture; 

preparing second sub-picture information 



(62) containing pixel data (PXD) constituting the 
sub-picture and being compr e ssed according to a 
predetermined manner, said second sub-picture 
information (62) constituting a bottom field of the 
one display frame of said sub-picture; 

preparing display control sequence informa- 
tion (33) containing one or more display control 
sequence information pieces (DCSQT; SPDCSQT) 
which control an order of display of the sub-picture 
based on said first and second sii>-picture informa- 
tion (61, 62), said display control sequence infor- 
mation pieces (DCSQT) Inducting display control 
start time information (SPDCTS), address informa- 
tion (SPNDCSQA) of subsequent display control 
sequences, and one or more display control com- 
mands (SPDCCMD); 

obtaining from said display control sequence 
in fo rm ation pieces (DCSQT) a first command (63) 
for setting a display start adcfress of pixels in the top 
field of said display frame; 

obtaining from said display control sequence 
information pieces (DCSQT) a second command 
(64) for setting a display start address of pixels in 
the bottom field of said display frame; 

preparing sub-picture header information 
(31 ; SPUH) containing size information (SPDSZ) of 
the sub-picture packet and location in forma t io n 
(SPDCSQTA) of said display control sequence 
information (33); and 

encoding each packet of the sub-picture in 
accordance with said prepared packet header infor- 
mation (3), said prepared first and second sub-pic- 
ture information (61, 62), said prepared display 
control sequence information (33). and said pre- 
pared sub-picture header information 01). 

54u A method of encoding image information wherein a 
sifo-plcture simultaneously reproducible with a 
main picture is encoded into one or more sub-pic- 
ture packets, said method comprising the steps of: 

preparing packet header information (PH in 
FIG. 3; 3 in FIG. 29) containing a time stamp (PTS 
in FIG. 28 or 30) which indicates a playback start 
time of the sub-picture packets; 

preparing sub-picture information (32 in FIG. 
3 or 29) containing pixel data (PXD) constituting the 
sub-picture and being compressed according to a 
predetermined manner; 

preparing display control sequence informa- 
tion (33 in FIG. 3 or 29) containing one or more dis- 
play control sequence information pieces (DCSQT 
in FIG. 3 or 29; SPDCSQT in FIG. 32) which control 
an order of cfsplay of the sub-picture based on said 
6ub-picture information (32); 

preparing sub-picture header information 
(31 in FIG. 3 or 29; SPUH in FIGL 3 or 31) contain- 
ing size Information (SPDSZ) of the sub-picture 
packet and location Information (SPDCSOTA) of 
said display control sequence information (33); 
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preparing pixel control data (PCD) for con- 
trolling at least one of color and contrast of the sub- 
picture during when trie sub-picture, corresponding 
to the compressed pixel data (PXD), is displayed; 
and 5 

encoding each packet of the sub-picture in 
accordance with said prepared packet header infor- 
mation (3), said prepared sub-picture information 
(32), said prepared display control sequence infor- 
mation (33), and said prepared sub-picture header w 
information (31). 

55. The method of daim 54, characterized in that said 
sub-picture is displayed in a sii>-picture frame con- 
stituted by a plurality of Ones, and said pixel control is 
data (PCD) includes: 

line control information (LCINF) for indicat- 
ing a specific line in which the color or the contrast 
of the pixel data changes; and 

pixel control information (PCI) for indicating 20 
a specific position, at which the color or the contrast 
changes, on the line indicated by said line control 
infor mati on (LCINF). 

56. A transmitting/receiving system wherein a sub-pic- 25 
ture simultaneously reproducible with a main pic- 
ture is encoded in unit of a packet, 6aid system 
comprising the steps of: 

converting (ST10) a content of the sub-pic- 
ture into bit-mapped sub-picture data; so 

preparing (ST 12) a spatial parameter 
(SETDAREA) for setting a display area of the sub- 
picture on a display area of the main picture; 

compressing (ST14) the bit-mapped sub- 
picture data based on a predetermined manner 60 55 
as to generate compressed pixel data (PXD); 

preparing a time parameter 
(STA_DSP/STP_DSP) for specifying display start 
timing and display stop timing of the sub-picture 
(ST16); 40 

preparing (ST18) one or more display con- 
trol sequence table (DCSQT) for controlling an 
older of display of the sub-picture; 

preparing (ST20) a sub-picture unit header 
(SPUH) containing size information (SPDSZ) of a 46 
packet of the sub-picture and location information 
(SPDCSQTA) of the display control sequence table 
(DCSQT), and combining the compressed pixel 
data (PXD) and the display control sequence table 
(DCSQT) with the sub-picture wiK header (SPUH) bo 
so as to provide a sub-picture unit (30); 

making (ST24) a packet of the sub-picture 
unit (30) when a size of the sub-picture unft (30) 
exceeds a predetermined value (2k bytes); 

recording (ST26) a time stamp (PTE) at the ss 
packet of the sub-picture unit (30) having a size 
exceeding the predetermined value (2k bytes), or at 
the sub-picture unit (30) having a size not more 
than the predetermined value (2k bytes), 6aid time 



stamp (PTS) representing a playback start time of 
the sub-picture; 

packing (ST28) the packet of said sub-pic- 
ture unit (30) having the lime stamp (PTS), or pack- 
ing (ST28) said sub-picture unit (30) having the 
time stamp (PTS), so as to make a data stream 
containing data of the packed sub-picture unit (30) 
and other data: 

transmitting the data stream; 

receiving the transmitted data stream; 

separating (ST40) the data of the packed 
sub-picture unit (30) from the received data stream; 

disassembling (ST44) the pack of the sepa- 
rated sub-picture unit 00); 

extracting (ST448) the time stamp (PTS) 
from the cfisassembled pack when the time stamp 
(PTS) is recorded at the cfisassenMed pack; and 

comparing (ST640) a content of the 
extracted time starrp (PTS) with time given by a 
system timer (120 in FIGL 11), checking (STB44) 
one or more contents of the display control 
sequence table (DCSQT) when the content of the 
extracted time stamp (PTS) matches the time given 
by the system timer (120), and expanding (STB46) 
the compressed pixel data (PXD) according to the 
checked contents of the display control sequence 
table (DCSQT). 

57. A transmitting/receiving system wherein a sub-pic- 
ture simultaneously reproducible with a main pic- 
ture is encoded in unit of a packet, said system 
comprising the steps of: 

compressing (ST14) a content of the sub- 
picture based on a predetermined manner so as to 
generate c omp r es sed pixel data (PXD); 

preparing a time parameter 
(STA_DSP/STP_DSP) for specifying display start 
timing and cf^lay stop timing of the sub-picture 
(ST16); 

preparing (ST18) one or more display con- 
trol sequence table (DCSQT) for controlling an 
order of display of the sub-picture; 

preparing (ST20) a sub-picture unit header 
(SPUH) containing size inform a tion (SPDSZ) of a 
packet of the sub-picture and location information 
(SPDCSQTA) of the display control sequence table 
(DCSQT), and combining the compressed pixel 
data (PXD) and the display control sequence table 
(DCSQT) with the sub-picture unit header (SPUH) 
so as to provide a sub-picture unit (30); 

making (ST24) a packet of the sub-picture 
unit (30) when a size of the sub-pictire unit (30) 
exceeds a predetermined value (2k bytes); 

recording (ST26) a lime stamp (PTS) at the 
packet of the subptafture unit (30) having a size 
exceeding the predetermined value (2k bytes), or at 
the sub-picture unit (30) having a size not more 
than the predetermined value (2k bytes), said time 
stamp (PTS) representing a playback start time of 
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the sub-picture; 

packing (ST28) the packet of said sub-pic- 
ture unit (30) having the time stamp (PTS), or pack- 
ing (ST28) said sub-picture unit (30) having the 
time stenp (PTS), so as to make a data stream 5 
containing data of the packed sub-picture unit (30) 
and other data; 

transmitting the data stream; 

receiving the transmitted data stream; 

separating (ST40) the data of the packed w 
sub-picture unit (30) from the received data stream; 

disassembling (ST44) the pack of the sepa- 
rated sub-picture unit (30); 

extracting (ST448) the time stamp (PTS) 
from the disassentied pack when the time stamp 75 
(PTS) is recorded at the disassembled pack; and 

comparing (ST640) a content of the 
extracted time stamp (PTS) with time given by a 
system timer (120 in FIG. 11), checking (STB44) 
one or more contents of the display control so 
sequence table (DCSQT) when the content of the 
extracted time stamp (PTS) matches the time given 
by the system timer (120). and expanding (ST646) 
the compressed pixel data (PXD) according to the 
checked contents of the display control sequence 25 
table (DCSQT). 

58. A method of encoding information wherein a sub- 
information simultaneously reproducible with a 
main i nforma ti o n is encoded into one or more sub- 30 
information packets, said method comprising the 
steps of: 

preparing packet header information (PH in 
FIG. 3; 3 in FIG. 29) containing a time stamp (PTS 
in FIG. 28 or 30) which intfcates an operation start ss 
time of the sub-information packets; 

preparing sub-information (32 in FIG. 3 or 
29) containing pixel data (PXD) constituting the 
sub-information and being compressed according 
to a predetermined manner; 40 

preparing control sequence information (33 
in FIG. 3 or 29) containing one or more control 
sequence information pieces (DCSQT in FIG. 3 or 
29; SPDCSQT in FIG. 32) which control an order of 
use of the sub-information based on said sub-infor- 45 
mation (32); 

preparing siJb-iiifbriTialJon header informa- 
tion pi in FIG. 3 or 29; SPUH in FIG. 3 or 31) con- 
taining size information (SPDSZ) of the sub- 
information pactot and location information (SPDG- so 
SQTA) of said control sequence information (33); 
and 

encoding each packet of the stiHnformation 
in accordance with said prepared packet header 
information (3), said prepared sub-information (32), bs 
said prepared control sequence information (33), 
and said prepared sii> information header informa- 
tion (31). 



59- A method of encoding information wherein a sub- 
informaiion simultaneously reprod u c ib le with a 
main information is encoded into one or more sub- 
information packets, said method corrprising the 
steps of: 

preparing packet header information (PH in 
FIG. 3; 3 in FIG. 29) containing a tome stamp (PTS 
in FIG. 28 or 30) which indicates an operation start 
time of the sub-information packets; 

preparing sub-information (32 in FIG. 3 or 
29) containing pixel data (PXD) constituting the 
sub-inforrralion and being processed according to 
a predetermined manner; 

preparing control sequence information (33 
in FIG. 3 or 29) containing one or more control 
sequence information pieces (DCSQT in FIG. 3 or 
29; SPDCSQT in FI& 32) which control an order of 
use of the sub-information based on said sub-infor- 
mation (32); 

preparing sub-i n form a tion header informa- 
tion (31 in Fia 3 or 29; SPUH in FIG. 3 or 31) con- 
taining size information (SPDSZ) of the sub- 
information packet and location in formati on (SPDC- 
SQTA) of said control sequence infor mati on (S3); 
and 

encoding each packet of the sub-information 
in accordance with said prepared packet header 
information (3), said prepared sub-information (32), 
said prepared control sequence information (33), 
and said prepared sutHrrformation header informa- 
tion (31). 

6a A method of encoding information wherein a sub- 
information simultaneously reproducible with a 
main information is encoded into two or more sub- 
information packets, 

(a) wherein, as for a first one of the sub-infor- 
mation packets, raid method comprises the 
steps of: 

preparing packet header information 
(PH in FIG. 3; 3 in FIG. 29) containing a torn 
stamp (PTS in FIG. 28 or 30) which irrfcates 
an operation start time of the sub-information 
packets; 

preparing sub-information (32 in FIG. 3 
or 29) containing pixel data (PXD) constituting 
the sub-i ifor mation and being compressed 
according to a predetermined manner; 

preparing control sequence information 
(33 in FIGL 3 or 29) containing one or more con- 
trol sequence information pieces (DCSQT in 
FIGL 3 or 29; SPDCSQT in FK2L 32) which con- 
trol an onter of use of the sub-information 
based on said sub-Information (32); and 

preparing sub-information header infor- 
mation (31 in FIG. 3 or 29; SPUH in FIG. 3 or 
31) containing size information (SPDSZ) of the 
sub-information packet and location informa- 
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tion (SPDCSQTA) of said control sequence 
information (33), and 

(b) wherein, as for a second and subsequent 
ones of the sub-information packets, said 
method comprises the steps of: s 

preparing sub-information (32 in FIG. 3 
or 29) containing pixel data (PXD) constituting 
the sub-information and being compressed 
according to a predetermined manner; 

preparing control sequence information to 
(33 in FIG. 3 or 29) containing one or more con- 
trol sequence information pieces (DCSQT in 
FIG. 3 or 29; SPDCSQT in FIG. 32) which con- 
trol an order of use of the sub-information 
based on said sub-information (32); and is 

preparing sub-inf or m a t io n header infor- 
mation (31 in FIG. 3 or 29; SPUH in FIG. 3 or 
31) containing size information (SPDSZ) of the 
sub-information packet and location informa- 
tion (SPDCSQTA) of said control sequence 20 
information (33). 

61. A method of encocfing information wherein a sub- 
information simultaneously reproducible with a 
main information is encoded into two or more sub- 2s 
information packets, 

(a) wherein, as for a first one of the sub-infor- 
mation packets, said method comprises the 
steps of: so 

preparing packet header information 
(PH in FIG. 3; 3 in RG. 29) containing a time 
stamp (PTS in FIG. 28 or 30) which indicates 
an operation start time of the sub-information 
packets; & 

preparing sub-information (32 in FIG. 3 
or 29) containing pixel data (PXD) constituting 
the sii>i nomi na tion and being processed 
according to a predetermined manner; 

preparing control sequence information 40 
(33 in FIG. 3 or 29) containing one or more con- 
trol sequence information pieces (DCSQT in 
FIG. 3 or 29; SPDCSQT in FIG. 32) which con- 
trol an order of use of the sub-information 
based on said sii>-irTformation (32); and 45 

preparing sii>information header infor- 
mation (31 in RG. 3 or 29; SPUH in FIG. 3 or 
31) containing size information (SPDSZ) of the 
sub-information packet and location informa- 
tion (SPDCSQTA) of said control sequence bo 
information (33), and 

(b) wherein, as for a second and subsequent 
ones of the sub-information packets, said 
method comprises the steps of: 

preparing sub-information (32 in FIG. 3 ss 
or 29) containing pixel data (PXD) constituting 
the sJ>4riformation and being processed 
according to a predetermined manner; 

preparing control sequence information 



(33 in FIG. 3 or 29) containing one or more con- 
trol sequence information pieces (DCSQT in 
FIG. 3 or 29; SPDCSQT In FIG. 32) which con- 
trol an oider of use of the sub-information 
based on said subnnformation (32); and 

preparing sub-information header infor- 
mation (31 in FIG. 3 or 29; SPUH in FIG. 3 or 
31) containing size information (SPDSZ) of the 
sub-information packet and location informa- 
tion (SPDCSQTA) of said control sequence 
information (33). 

62. A recording medium for recording one or more sub- 
information packets containing a si±>-mformalion 
simultaneously reproducible with a main informa- 
tion, said record ng medium containing: 

packet header information (PH in FIG. 3; 3 in 
FIG. 29) containing a time stamp (PTS in FIG. 28 or 
30) which indicates an operation start time of the 
sub-information packets; 

sub-information header information (31 in 
FIG. 3 or 29; SPUH in FIG. 3 or 31 ) containing size 
information (SPDSZ) of the sub-information packet; 

sub-information (32 in FIG. 3 or 29) contain- 
ing pixel data (PXD) constituting the sub-informa- 
tion and being compressed according Id a 
predetermined manner; and 

control sequence information (33 in FIG. 3 or 
29} containing one or more control sequence infor- 
mation pieces (DCSQT in FIG. 3 or 29; SPDCSQT 
in FIG. 32) which control an older of use of the sub- 
information based on wxi sub-information (32). 

63. A recording medium for recording one or more sub- 
in formation packets containing a sub-information 
simuHaneously reproducible with a main informa- 
tion, said recording medium containing: 

packet header information (PH in RG. 3; 3 in 
FIG. 29) containing a lime stamp (PTS in FIG. 28 or 
30) which indicates an operation start time of the 
sub-information pac&Bts; 

sub-information header information (31 in 
Fia 3 or 29; SPUH in FIG. 3 or 31) containing size 
information (SPDSZ) of the sub-information packet; 

sub-information (32 In FIG. 3 or 29) contain- 
ing pixel (tela (PXD) constituting the sub-informa- 
tion and being processed according to a 
predetermined manner; and 

control sequence information (33 in FIG. 3 or 
29) containing one or more control sequence Infor- 
mation pieces (DCSQT in FIG. 3 or 29; SPDCSQT 
in FIG. 32) which control an order of use at the sub- 
information based on said sub-information (32). 

64. An encocCngtiecoding system wherein a sii>-intar- 
mation simultaneously reproducfoie with a main 
information is encoded in unit of a packeL said sys- 
tem comprising the steps of: 

converting (ST10) a content of the sub-irrfor- 
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mation into bil-mapped sub- information data; 

preparing (ST12) a spatial parameter 
(SETJDAREA) for setting a use area of the sii> 
intormation on a use area of the main information; 

compressing (ST14) the bit-mapped sub- s 
information data based on a predetermined manner 
so as to generate compressed pixel data (PXD); 

preparing a time parameter 
(STA_DSP/STP_DSP) for specifying use start tim- 
ing and use stop taming of the sub-information w 
(ST16); 

preparing (ST18) one or more control 
sequence table (DCSQT) for controlling an order of 
use of the sub-information; 

preparing (ST20) a sub-information unit 75 
header (SPUH) containing size information 
(SPDS2) of a packet of the sub-iritorrnation and 
location information (SPDCSQTA) of the control 
sequence table (DCSQT), and combining the com- 
pressed pixBl data (PXD) and the control sequence so 
table (DCSQT) with the sub-information unit header 
(SPUH) so as to provide a siiwnrformafion unit 
(30); 

making (ST24) a packet of the sub-informa- 
tion unit (30) when a size of the sub-information unit & 
(30) exceeds a predetermined value (2k bytes); 

recording (ST26) a time stamp (PTS) at the 
packet of the sub-information unit (30) having a size 
exceeding the predetermined value (2k bytes), or at 
the sub-information unit (30) having a size not more so 
than the predetermined value (2k bytes), said time 
stamp (PTS) representing a use start time of the 
sub-information; 

packing (ST28) the packet of sad sub-infor- 
mation unit (30) having the time stamp (PTS), or 3s 
packing (S728) said sub-information unit (30) hav- 
ing the lime starrp (PTS), so as to make a data 
stream containing data of the packed sub-informa- 
tion unit (30) and other data; 

separating (ST40) the data of the packed 40 
sub-irrfonnalion unit (30) from the data stream; 

disassembling (ST44) the pack of the sepa- 
rated sub-information unit (30); 

extracting (ST448) the time stanp (PTS) 
from the disassembled pack when the lime stamp 45 
(PTS) Is recorded at the disassembled pack; and 

comparing (ST640) a content of the 
extracted tvne stamp (PTS) with time given by a 
system timer (120 in FIG. 11), checking (ST644) 
one or more contents of the control sequence table so 
(DCSQT) when the content of the extr a cted time 
stamp (PTS) matches the time given by the system 
timer (120), and expanding (S7B46) the com- 
pressed pixel data (PXD) aocorcfing to the checked 
contents of the control sequence table (DCSQT). ss 

65. An encoding/decoding system wherein a sub-infor- 
mation simultaneously reproducible with a main 
information is encoded in unit of a packet, said sys- 



tem comprising the steps of: 

converting (ST1 0) a content of the sub-infor- 
mation into bit-mapped sub-information data; 

preparing (ST12) a spatial parameter 
(5ET_DAREA) for setting a use area of the sub- 
intormation on a use area of the main information; 

processing (ST14) the bit-mapped sub-infor- 
mation data based on a predetermined manner so 
as to generate processed pixel data (PXD); 

preparing a time parameter 
(STAJDSP/3TP_DSP) for specifying use start tim- 
ing and use stop timing of the siinnformation 
(ST16); 

preparing (ST18) one or more control 
sequence table (DCSQT) for controlling an order of 
use of the sub-information; 

preparing (ST20) a sub-information unit 
header (SPUH) containing size information 
(SPDSZ) of a packet of the sub-information and 
location information (SPDCSQTA) of the control 
sequence table (DCSQT), and combining the proc- 
essed pixel data (PXD) and the control sequence 
table (DCSQT) with the sub-information unit header 
(SPUH) so as to provide a sub-information unit 
(30); 

making (ST24) a packet of the sub-informa- 
tion unit (30) when a size of the sub-information unit 
(30) exceeds a predetermined value (2k bytes); 

recording (ST26) a time starrp (PTS) at the 
packet of the sub-information unit (30) having a size 
exceeding the predetermined value (2k bytes), or at 
the sub-information unit (30) having a size not more 
than the predetermined value (2k bytes), said time 
stamp (PTS) representing a use start time of the 
sub-information; 

packing (ST28) the packet of said sub-infor- 
mation unit (30) having the time stamp (PTS), or 
packing (ST28) said sub-information unit 00) hav- 
ing the time stamp (PTS), so as to make a data 
stream containing data of the packed sub-informa- 
tion unit (30) and other data; 

separating (ST40) the data of the packed 
suMnrlbrmation urnt (30) from the data stream; 

disassembling (ST44) the pack of the sepa- 
rated sub-information unit (30); 

extracting (ST448) the time stamp (PTS) 
from the disassembled pack when the time stamp 
(PTS) Is recorded at the disassembled pack; and 

conparing (ST640) a content of the 
extracted time stamp (PTS) with time given by a 
system timar (120 in RG. 11). checking (ST644) 
one or more contents of the control sequence table 
(DCSQT) when the content of the extracted time 
stamp (PTS) matches the time given by the system 
timer (120), and expand ng (ST646) the processed 
pixel data (PXD) according to the checked contents 
of the control sequence table (DCSQT). 
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66. A broadcasting system for encoding and broadcast- 
ing i nform ation wherein a sub-information simulta- 
neously reproducible with a main information Is 
encoded into one or more sub-information packets, 
said broadcasting system comprising: $ 

means (ST26) for preparing packet header 
information (PH/3) containing a time stamp (PTS) 
which indicates a use start time of the sub-informa- 
tion packets; 

means (ST14) for preparing sub-irfforrnatton w 
(32) which contains pixel data (PXD) constituting 
the siiMnformatton and being compressed accord- 
ing to a predetermined manner; 

means (ST12, ST16, ST18) for preparing 
control sequence information (S3) containing one or rs 
more control sequence i nformatio n pieces 
(DCSQT/SPDCSQT) which control an order of use 
of the sub-information using said sub-information 
(32); 

means (ST20) for generating each unit of the 20 
sub-information packets in accordance with said 
packet header information (3), said sub-information 
(32), said control sequence information (33), and 
sub-information header information (31) which con- 
tains size information (SPDSZ) ol the sub-informa- 25 
tion packet and location information (SPDCSQTA) 
of the control sequence information (33); and 

means (FIG. 22) for broadcasting a data 
stream containing the generated sub-information 
packets. 30 

67. A broadcasting system for encoding and broadcast- 
ing information wherein a sub-information simulta- 
neously reproducible with a main information is 
encoded into one or more sub-information packets, 35 
said broadcasting system comprising: 

means (ST26) for preparing packet header 
information (PH/3) containing a time stamp (PTS) 
which indicates a use start time of the sub-informa- 
tion packets; & 

means (ST14) for preparing cub-information 
(32) which contains pixel data (PXD) constituting 
the sub-information and being processed according 
to a predetermined manner; 

means (ST12. ST16, ST18) tor preparing 4S 
control sequence information (33) containing one or 
more control sequence information pieces 
(DCSQT/SPDCSQT) which control an order of use 
of the sub-information using said sub-infcx*mation 
(32); so 

means (ST20) for generating each unit of the 
sub-information packets in accordance with said 
packet header information (3), said conformation 
(32), said control sequence information (33), and 
sub-information header information (31) which con- bs 
tains size information (SPDSZ) ol the sub-informa- 
tion packet and location information (SPDCSQTA) 
of the control sequence information (33); and 

means (FIG. 22) for broadcasting a data 



stream containing the generated sub-information 
packets. 

68. An apparatus of decorJng a data stream obtained 
by packing a plurality of sub-information units (30) 
each of which includes: 

a time stamp (PTS) indicating a use start 
time of bit-mapped sub-information packets; 

control sequence table (DCSQT) tor control- 
ling an order of use of the sub-iirfon nation; 

pixel data (PXD) obtained by compressing 
the bit-mapped sub-ffiformation; and 

sub-information unit header (SPUH) contain- 
ing size information (SPDSZ) of the sub-information 
packet and location information (SPDCSQTA) of 
the control sequence table (DCSQT), said appara- 
tus comprising: 

means for separating (ST40) the data of the 
packed sub-inforrnation unit (30) from the data 
stream; 

means for disassembling (ST44) the pack of 
the separated sub-Information mit (30); 

means tor extracting (ST448) the time stamp 
(PTS) from the disassembled pack when the time 
stamp (PTS) is recorded at the disassembled pack; 
and 

means (112) for comparing (ST640) a con- 
tent of the extracted time stamp (PTS) with time 
given by a system timer (120 in FIG. 11), checking 
(ST644) one or more contents of the control 
sequence table (DCSQT) whan the content of the 
extracted time s tamp (PTS) matches the time given 
by the system timer (120), and expanding (STB46) 
the compressed pixel data (PXD) accorrfing to the 
checked contents of the control sequence table 
(DCSQT) so as to provide non-encoded sii>infor- 
mation. 

69. An apparatus of decocfng a data stream obtained 
by packing a plurality of sub-information units (30) 
each of which includes: 

a time stamp (PTS) indicating a use start 
time of bit-mapped sub-information packets; 

control sequence table (DCSQT) tor control- 
ling an order of use of the sub-information; 

pixel data (PXD) obtained by processing the 
bit-mapped sub-in formation ; and 

sub-information unit header (SPUH) contain- 
ing size information (SPDSZ) of the sub-information 
packet and location intormetion (SPDCSQTA) of 
the control sequence table (DCSQT), said appara- 
tus comprising: 

means for separating (ST40) the data of the 
packed sub-information unit (30) from the data 
stream; 

means for disassembling (ST44) the pack of 
the separated sub-information \mv\ (30); 

means for extracting (ST448) the time stamp 
(PTS) from the disassembled pack when the time 
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stamp (PTS) is recorded at the disassembled pack; 
and 

means (1 12) for comparing (ST640) a con- 
tent of the extracted time stamp (PTS) with time 
given by a system timer (1 20 "m FIGL 1 1), checking 5 
(ST644) one or more contents of the control 
sequence table (DCSQT) when the content of the 
extracted time stamp (PTS) matches the tone given 
by the system timer (120), and expanding (ST646) 
the processed pixel data (PXD) according to the 717 
checked contents of the control sequence table 
(DCSQT) so as to provide non-encoded sub-infor- 
mation. 

70. A transmitting/receiving system wherein a sub- is 
information simultaneously reproducible with a 
main information is encoded in unit of a packet, said 
system comprising the steps of: 

compressing (ST14) a content of the sub- 
information based on a predetermined manner so so 
as to generate compressed pixel data (PXD); 

preparing a time parameter (STA_DSP/- 
STP_DSP) for specifying use start timing and use 
stop timing of the sub-information (ST16); 

preparing (ST18) one or more control ss 
sequence table (DCSQT) for controlling an order of 
use of the sub-information; 

preparing (ST20) a sub-information unit 
header (SPUH) containing size information 
(SPDSZ) of a packet of the sub-information and so 
location information (SPDCSQTA) of the control 
sequence table (DCSQT), and combining the com- 
pressed pixel data (PXD) and the control sequence 
table (DCSQT) with the sub-information unit header 
(SPUH) so as to provide a sub-irrformalion urvt as 
(30); 

making (ST24) a packet Of the sub-informa- 
tion unit (30) when a size of the sub-information unit 
(30) exceeds a predetermined value (2k bytes); 

recording (ST26) a time stamp (PTS) at the 40 
packet of the sub-information unit (30) having a size 
exceeding the predetermined value (2k bytes) , or at 
the sub-information unit (30) having a size not more 
than the predetermined value (2k bytes), said lime 
stamp (PTS) representing a playback start time of 45 
the sub-Information; 

packing (ST28) the packet of said sub-infor- 
mation unit (30) having the time stamp (PTS), or 
packing (ST28) 6aid sub-information unit (30) hav- 
ing the time stamp (PTS), so as to make a data bo 
stream containing data of the packed sub-informa- 
tion init (30) and other data; 

transmitting the data stream; 

receiving the transmrtted data stream; 

separating (ST40) the data of the packed ss 
sub-information unit (30) from the received data 
stream; 

disassembling (ST44) the pack of the sepa- 
rated sub-i nfor m ati on unit (30); 



extracting (ST448) the time stamp (PTS) 
from the disassembled pack when the time stamp 
(PTS) is recorded at the disassembl ed pack; and 

conparing (ST640) a content of the 
extracted time stamp (PTS) with time given by a 
system timer (120 in FIG. 11), checking (ST644) 
one or more contents of the control sequence table 
(DCSQT) when the content of the extracted time 
stamp {PTS) matc hes the time given by the system 
timer (120), and expanding (ST646) the com- 
pressed pixel data (PXD) according to the checked 
contents of the control sequence table (DCSQT). 

71. A transmitting/receiving system wherein a sub- 
information simultaneously reproducible with a 
main information is encoded in unit of a packet said 
system comprising the steps of; 

processi ng (ST1 4) a content of the sub-infor- 
mation based on a predetermined manner so as to 
generate processed pixel data (PXD) ; 

preparing a time parameter (STA_DSP/- 
STP_DSP) for specifying use start timing and use 
stop timing of the sub-information (ST16); 

preparing (ST18) one or more control 
sequence table (DCSQT) for controlling an order of 
use of the sub-friformation; 

preparing (ST20) a sub-information unit 
header (SPUH) containing size information 
(SPDSZ) of a packet of the sub-information and 
location i nfo rm a t io n (SPDCSQTA) of the control 
sequence table (DCSQT), and combkiing the proc- 
essed pixel data (PXD) and the control sequence 
table (DCSQT) with the sub-information unit header 
(SPUH) so as to provide a sub-information unit 
(30); 

making (ST24) a packet of the siiHnfbrma- 
tion unit 00) when a size of the sub-information unit 
(30) exceeds a predetermined value (2k bytes); 

record ng (ST26) a time stamp (PTS) at the 
packet of the sub-information unit (30) having a size 
exceeding the predetermined value (2k byte), or at 
the sub-information unit (30) having a size not more 
than the predetermined value (2k bytes), said time 
stamp (PTS) representing a playback start time of 
the sub-I n for ma tion; 

packing (ST28) the packet of said sub-infor- 
mation unit (30) having the time stamp (PTS), or 
packing (ST28) said sub-information unit (30) hav- 
ing the time stamp (PTS), so as to make a data 
stream containing data of the packed sub-informa- 
tion unit (30) and other data; 

transmitting the data stream; 

recelvfrig the transmitted data stream; 

separating (ST40) the data of the packed 
stiHirfbrmation unit (30) from the received data 
stream; 

disassembling (ST44) the pack of the sepa- 
rated sii)-frifc>rmaiiori unit (30); 

extracting (ST448) the time stamp (PTS) 
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from the disassembled pack when the time stamp 
(PTS) is recorded at the disassembled pack; and 

comparing (ST640) a content of the 
extracted time stamp (PTS) whh time given by a 
system timer (120 in FIGL 11), checking (ST644) s 
one or more contents of the control sequence table 
(DCSQT) when the content of the extracted time 
stamp (PTS) matches the time given by the system 
timer (120), and expanding (ST646) the processed 
pixel data (PXD) according to the checked contents w 
of the control sequence table (DCSQT). 
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RESERVED(O) 
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END Y-COORDINATE (LOWER BITS) 
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COMMAND "SET DSPXA(06h) M FOR SETTING 
DISPLAY START ADDRESS OF PIXEL DATA 
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FOR TOP FIELD (UPPER BITS) 
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HEAD ADDRESS OF PIXEL DATA 






FOR TOP FIELD (LOWER BITS) 
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HEAD ADDRESS OF PIXEL DATA 
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HEAD ADDRESS OF PIXEL DATA 








FOR BOTTOM FIELD (LOWER BITS) 
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COMMAND "CHG COLCON(07h)" FOR CHANGING 
COLOR/CONTRAST FOR PIXEL DATA 

bm bm-1 bm-2 bm-3 bm-4 bm-5 bm-6 bm-7 



0 


0 


0 


0 


0 


1 


0 


1 



bm-8 bm-9 bm-1 0 bm-1 1 bm-1 2 bm-1 3 bm-1 4 bm-1 5 



EXTENDED FIELD SIZE (UPPER BITS) 



bm-1 6bm-1 7 bm-1 8 bm-1 9bm-20bm-21 bm-22bm-23 
EXTENDED FIELD SIZE (LOWER BITS) 



bm-24bm-25bm-26 bm-27bm-28bm-29 bm-30bm-31 
PIXEL CONTROL DATA PCD (START) 



b7 b6 b5 b4 b3 b2 bl bO 
PIXEL CONTROL DATA PCD (END) 



FIG. 44 

COMMAND "CMD END(FFh)" FOR ENDING 
DISPLAY CONTROL OF PIXEL DATA 
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LINE CONTROL INFORMATION LCINF 






b31 b30 b29 b28 


b27 b26 


b25 b24 




RESERVED (0) 


CHANGE START 
LINE NUMBER 
(UPPER BITS) 


b23 b22 b21 b20 


b19 b18 


b17 b16 




CHANGE START LINE NUMBER (LOWER BITS) 




b15 b14 b!3 bl2 


b11 blO 


b9 b8 




NUMBER OF CHANGES 


RESERVED(O) 


CHANGE TERMI- 
NATION UNE NO. 
(UPPER BITS) 


b7 b6 b5 b4 


b3 b2 


b1 bO 





CHANGE TERMINATION LINE NUMBER (LOWER BITS) 



CHANGE START LINE NUMBER; NUMBER OF LINE FROM WHICH 

CONTENT OF PIXEL CONTROL 
STARTS TO CHANGE; 
DESCRIBED BY LINE NUMBER 
OF VIDEO DISPLAY 

NUMBER OF CHANGES; NUMBER OF CHANGE POINTS 

(NUMBER OF PCINFs) SPREAD 
EVENLY ON LINES TO BE CHANGED 
COULD BE DESCRIBED WITH 
NUMBERS BETWEEN 1 AND 8. 

CHANGE TERMINATION LINE NO.; NUMBER OF LINE AT WHICH 

CONTENT OF PIXEL CONTROL 
TERMINATES THE CHANGE; 
DESCRIBED BY LINE NUMBER 
OF VIDEO DISPLAY 
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PIXEL CONTROL INFORMATION PCINF 
h47 b46 b45 b44 b43 b42 



b41 b4Q 



RESERVED CO) 


CHANGE START 
PIXEL NUMBER 
(UPPER BITS) 


b39 b38 b37 b36 


b3S 


b34 


b33 b32 


| CHANGE START PIXEL NUMBER (LOWER BITS) | 


b31 b30 b29 b28 


b27 


b26 


b25 b24 


^JEW EMPH. PIX.2 COL- CODE | 


NEW EMPH. PIX.1 COL CODE | 


b23 b22 b21 b20 


b19 


b!8 


b17 b16 


|NEW PATRN PIX. COL. CODE 


NEW BKGND PIX. COL CODE | 


b15 b14 b13 b12 


bll 


blO 


b9 b8 


NEW EMPH. PIX.2 CONTR. 


.NEW EMPH. PIX.1 CONTR. 




b7 b6 b5 b4 


b3 


b2 


bl bO 





NEW PATRN PIX. CONTR. 



NEW BKGND PIX. CONTR. 



CHANGE START PIX. NO.; NUMBER OF PIX. FROM WHICH CONTENT OF 

PIX. CONTROL STARTS TO CHANGE, & 
DESCRIBED BY PIX. NO. OF VIDEO DISPLAY 
NEW EMPH. PIX.1 &2 COLOR; DESCRIBES PALETTE CODE FOR 

EMPHASIS PIXEL 1&2 ON CHANGE START 
PIXEL & FOLLOWING PIXELS 
NEW PATRN PIX. COLOR; DESCRIBES PALETTE CODE FOR 

PATTERN PIXEL ON CHANGE START 
PIXEL & FOLLOWING PIXELS 
NEW CONTRAST OTHER THAN BACKGROUND PIXELS; DESCRIBES 

CONTRAST FOR EMPHASIS PIXELS 1 &2 
AND FOR PATTERN PIXEL ON CHANGE START 
PIXEL & FOLLOWING PIXELS 
NEW CONTRAST FOR BACKGROUND PIXELS; DESCRIBES CONTRAST FOR 

BACKGROUND PIXEL ON CHANGE START 
PIXEL & FOLLOWING PIXELS 
NOTE; ENTER SAME CODE AS INITIAL VALUE 

IF NO CHANGE IS REQUIRED 
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POS. 



CHANGE 
LINE NO. 
= 4 




1 23456789 1011112131 

LINEOO !OOi&6i6TO!b1( 
UNE01 lOiOIOlOiOiOIOK 

LINE02 loioiOiOloiaok 

lineo3 jo©ioroiordiOi< 
lineo4 iolo]6|6]6ia6bjoioi( 

LINE05 
LINE06 
LINE07 
LINE08 

LINE09 IO;O|.O;0|0:Oi»=«il 
LINE1~ ' " ' ' 

LINE 
LINE 12 
LINE! 3 



LINE14:OIO|0!0:6T6:< 

LINE15 l — 

LINE! 
LINE1 
LINE1 

LINE19 :OiO:0;0;0iO:O:O:l OiO; 
LINE20 ipjOlOiOlOipiOioll \0\Oi 

| (10PIX) — » 

j POS. D 

i (13 PIX) — - 



CONTROL 
ISTART 
LINE 



SitlJKSS-iSlBI control) 



POS. E 



6;6V6[o[6T6ib:bi6l 
oiPiQiojolo jo !p jp! 

\DISPLAY 
FRAME 
IMAGE 
OF SP 



NO. OF 
CONTI- 
NUOUS 
LINES 

= 7 



O SP DATA OF BIT "0"; 

•A®D**= SP DATA OF BIT "1 ■ FOR DIFFERENT OUTLINE 

COMPENSATION COLOR, SP COLOR & CONTRAST; 
CHANGE UNE « LINE ON WHICH BIT "1 " (NOTO) 

CHANGES TO OTHER BIT "1 " (NOTO) 
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PIXEL CONTROL DATA PCD 



PARAMETER 


CONTENTS 


POSI- 
TION 


LCINF1 


| CHANGE LINE NO. (04 TO 1 1 ) 




PCINF1 


; CHANGE START PIX. NO. 1 1 (LINE08) \ 


C 


PCINF2 


\ CHANGE START PIX. NO. 1 4 (LINE04) 


B 


PCINF3 


I CHANGE START PIX. NO. 1 7 (LINE 1 1 )"' 


A 


LCINF2 


CHANGE LINE NO. (12) 




PCINF1 


| CHANGE START PIX. NO. 09 (LINE! 2) 

! , 


D . 


PCINF2 


CHANGE START PIX. NO. 1 2 (LINE1 2) 


! E 


i 


l i 

t i 
i I 
i t 
t i 
l i 


; 


LCiNFm-1 


'CHANGE LINE NO. (14) 




PCINF1 


CHANGE START PIX. NO. 09 (LINE! 4) 


D 


PCINF2 


CHANGE START PIX. NO. 1 1 (LINE1 4) 


c 


PCINF3 , 

< 


CHANGE START PIX. NO. 1 2 (LINE1 4) 

h - 


! E 


PCINF4 


CHANGE START PIX. NO. 1 7 (LINE1 4) 


; A 


LCINFm 


CNAGE LINE NO. (14) 
(TERMINATION CODE) 
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:IMAGE OF DISPLAYED SUB-PICTURE DATA> 
!-* W 



;aO.QMaOQO.OM«|ji.QaO«QQQQ.O:0 

■QD;o.«MQD.aa«atto.Q.o;a.tQ.QQi.Q 
;o;Qa»ti.«.Qq.Qt;.Q»i.QaaaQii*.QQ 

:a.o:o.tM«oQo;i.« / *to.Q.QQiaQQto 



f 

H 

I 



0= BIT-MAPPED DATA OF A COLOR DISPLAY FRAME 

•= BIT-MAPPED DATA OF ANOTHER COLOR 

FIG. 50 



CONTINUOUS DISPLAY 
PERIOD OF SAME SP (NO. 6) 

DISPLAY 

frame □□□□E303 OEEEEEEinmn- 

DISPLAY tttttt ttttttttt 

FRAME 03 04 05 06 07 08 28 29 30 31 32 33 34 35 36 — 

NUMBERS 

DATA . . ~ ~ ^ „ _ ^ „ „ „ 

trans- O®0®(§SIS : SS®SS : Sss3®0 

MISSION ' SUBSTANTIALLY J 

CYCLE OF 
SP DISPLAY 



WASTEFUL DATA 



EXAMPLE OF SP DATA DISPLAYED 
ACCORDING TO DISPLAY FRAME NO. 06 
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STORED STATE OF SP BUFFER 
(BUFFERING AMOUNT OF SP) 



DATA BLOCK 
OF ONE SP 




*1 


*2 


*1 


*3 


>4f 


*2 


*1 


*5 


*4f 


*5 


*1 


*4 


*2 


'4f 


*3 


*1 



PACKET 
TRANS- 
MITTING 
ORDER 



PTD EXISTS PTD EXISTS 



PTD EXISTS 



*1 = VIDEO PACKET 

*2 - AUDIO PACKET 

*3 = 1ST CHANNEL OF SP PACKET 

*4f= 2ND CHANNEL OF SP PACKET (WITH PTS) 

*4 « 2ND CHANNEL OF SP PACKET (WITHOUT PTS) 

*5 = 3RD CHANNEL OF SP PACKET 
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^ START ^ 



PREPARE BIT-MAPPED DATA OF IMAGE/CAPTION / 
CORRESPONDING TO CONTENTS OF VIDEO A 



I 



ST10 



ST12 



PREPARE SPATIAL PARAMETERS OF DCSQ: ~Y 
DETERMINE PARAM. OF COMMAND "SET DAREA" 



I 



ENCODE PXD: 

RUN-LENGTH COMPRESSION OF 

BIT-MAPPED SP DATA 
DETERMINE PARAM. OF "SET COLOR" 
DETERMINE PARAM. OF ,, SETCONTR ,, 



I 



ST 14 



PREPARE TIME PARAMETERS OF DCSQ: 
DETERMINE MAX. TIME LIMIT OF PTS 
DETERMINE PARAM. OF "STA DSP" 
DETERMINE PARAM. OF "STP DSP" 
DETERMINE PARAM. OF "CHG COLCON" 



I 



ST16 
ST18 



PREPARE DCSQ REPRESENTING CONTENTS OF DCSQT 



I 



PREPARE SP UNIT (SPUH + PXD + DCSQT) 





UNIT SIZE > 2kB ? 
NO | 



PACKETING FOR ONE 



ST26 




ST20 
ST24 



PACKETING FOR 
MORE THAN ONE 

1 



RECORD PTS 



I 



ST28. 



PACK PACKETS OR UNITS OF SP; ASSEMBLE PACK & 
VIDEO, etc. TO GENERATE DATA STREAM 

E 



C END ) 
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DISASSEMBLING 
FOR PACK 

f START ) 

I 



DECODING 
FOR SP 

^ START 



DISASSEMBLE 
STREAM 



ST44 



ST40 



ST42 



DISASSEMBLING 
FOR i-TH PACK 



ST46 



TEMPORARY 
STORAGE OF 
DISASSEMBLED 
PACK AT 
MEMORY 1 08 



ST50 



ST52 



FINAL PACK 
YES 



NO 



( *° ) 



ST63 




1 -* j 



READ DATA 
OF j-TH PACK 
FROM 

MEMORY 1 08 



I 



j-TH PACK 
IS STORED IN 
MEMORY ? 



ST64 



YES 



DECODING 
FOR j-TH SP 



PARALLEL 
PROCESSING 



ST66 




ST60 



NO 



ST67 



END COMMAND^ 
EXECUTED ? 

YES 



( END ) 



NO 
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f ENTER ) 
1 



ST444. 



ST448 \ 



DISASSEMBLING 
FOR PACK 

PTS EXISTS ? 



YES 



EXTRACT ONLY PTS FROM 
PACKET HEADER & STORE IT 
IN BUFFER OF SP DECODER 



ST442 



NO 



ST446 



Q RETURN J 
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DELETE PACKET 
HEADER & STORE 
DISASSEMBLED 
DATA 



I 



Q ENTER ^ 
I 



COMPARE PTS WITH SYSTEM TIME! 



ST642 



T640 



^ MATCH? ^ > 



NO 



YES I 



EXECUTE COMMANDS OF DCSQ: 

SET DISPLAY POS./AREA BY "SET DAREA" 
SET DISPLAY COLOR BY "SET COLOR" & 
SET CONTRAST OF SP BY "SET CONTR" 



I 



DECODE SP PIXEL DATA PXD WHILE PERFORMING 
DISPLAY CONTROL BASED ON "CHG COLCON" 
UNLESS "STP DSP" IS EXECUTED IN OTHER DCSQ 
AFTER EXECUTION OF "STA DSP" 



ST644 



ST646 



Q RETURN J 
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^ START ^) 



ST40 



DISASSEMBLE 
STREAM 



ST42 



ST44 



DiSASSEMBLIN 
FOR i-TH PACK 



A 



ST46 



TEMPORARY 
STORAGE OF 
DISASSEMBLED 
PACK AT 
MEMORY 108 
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ST50 
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PARALLEL 
PROCESSING 



READ 
DATA OF 
j-TH PACK 
FROM 

MEMORY 108 
1 



DECODING 
FOR j-TH SP 



NO 



ST52 



FINAL PACK ? 



| YES 



DISASSEMBLING 
FOR PACKS 



ST66 




END COMMAND 
EXECUTED? 



YES 



( °* ) 



DECODING 

FOR SUB-PICTURE 
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COMMAND "SET DSPXA" (4 BYTES) 



~j TOP FIELD START ADDRESS (2 BYTES) 

i ■ > 

~\ BOTTOM FIELD START ADDRESS 
I (2 BYTES) : 



63 
64 



.61 



SP DATA (PXD) 
OF TOP FIELD 



62 



SP DATA (PXD) 
OF BOTTOM FIELD 



FIG. 58 



91 



EP 0725 541 A2 



DISPLAY 
CONTROL 
SEQUENCE 
TABLE 



DCSQTO 



.COMMAND 3(2) 
-COLOR DATA(2) 



COMMAND 4(2) 
■COLOR DATA(2) 



DCSQTX 



DATA PACKET 
OF NEXTSP"" 



SPDCTS(4) 
SPNDCSQA(2) 



COMMAND 5(2) 
DISPLAY START? 
POS. Xs, Ys & 
DISPLAY END 
POS. Xe, Ye 



COMMAND 6(2) 
LINE TABLE 
SIZE (2) 



DISPLAY LINE 
DATA POINTER 



fc6MMAND"7(2) 
PCDSIZE(2) 



DCSQ POINTER 



LINE TABLE 
POINTER 

(1ST LOCATION) 



LINE CONTROL 
FILE 0 (LCINF) 



PIX. CONTROL FILE 0~ 
PIX. CONTROL FILE N 



PCD POINTER 



END CODE 



SPDCTS(4) 
SPNDCSQA(2) 



END CODE 
~SPDSZ(2) 
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